2024-04-24T17:45:54.366 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-24T17:45:54.366 DEBUG:teuthology.run:Teuthology command: teuthology --name yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi --description rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writearound} --verbose --owner scheduled_yuriw@teuthology --archive /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641 -- /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/orig.config.yaml 2024-04-24T17:45:54.416 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T17:45:54.496 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641 branch: wip-yuriw4-testing-20240423.151325-quincy description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writearound} email: yweinste@redhat.com first_in_suite: false job_id: '7671641' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - machine: cpus: 1 disk: 40 ram: 8000 volumes: count: 4 size: 30 os_type: rhel os_version: '8.6' overrides: admin_socket: branch: wip-yuriw4-testing-20240423.151325-quincy ceph: conf: client: rbd cache: true rbd cache policy: writearound global: mon client directed command retry: 5 mon warn on pool no app: false ms inject socket failures: 5000 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bluestore block size: 96636764160 bluestore compression algorithm: lz4 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd objectstore: bluestore osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(CACHE_POOL_NEAR_FULL\) - \(CACHE_POOL_NO_HIT_SET\) - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: default sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuriw4-testing-20240423.151325-quincy sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 owner: scheduled_yuriw@teuthology priority: 99 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 9954 sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 sleep_before_teardown: 0 subset: 111/120000 suite: rbd suite_branch: wip-yuriw4-testing-20240423.151325-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 targets: smithi049.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAmpKeVOaV+kk9DAKzd4G75t9zj75Y7RbtC5dbT4qkkJnSiDmlEp5teDAKuw9Y2Pahlg5fVaJUzSKw+9SIOVJkg= smithi078.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfZwGy4KggpIQ5Xr3FpzGaV92di5pOwGC7vllv/74eSJo0Lobhx/D49PoG7Ya/JvdGWv84/cQGpAO67zRImDz4= tasks: - install: null - ceph: null - exec: client.0: - sudo ceph osd pool create cache 4 - sudo ceph osd tier add rbd cache - sudo ceph osd tier cache-mode cache writeback - sudo ceph osd tier set-overlay rbd cache - sudo ceph osd pool set cache hit_set_type bloom - sudo ceph osd pool set cache hit_set_count 8 - sudo ceph osd pool set cache hit_set_period 60 - sudo ceph osd pool set cache target_max_objects 250 - thrashosds: timeout: 1200 - rbd_fsx: clients: - client.0 ops: 6000 teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-24_14:09:24 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-24T17:45:54.496 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa; will attempt to use it 2024-04-24T17:45:54.497 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa/tasks 2024-04-24T17:45:54.497 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-24T17:45:54.498 INFO:teuthology.task.internal:Checking packages... 2024-04-24T17:45:54.523 INFO:teuthology.task.internal:Checking packages for os_type 'rhel', flavor 'default' and ceph hash '330067dbce47655d1f762ef14d546d2fde16a6a9' 2024-04-24T17:45:54.523 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-24T17:45:54.523 INFO:teuthology.packaging:ref: None 2024-04-24T17:45:54.523 INFO:teuthology.packaging:tag: None 2024-04-24T17:45:54.523 INFO:teuthology.packaging:branch: wip-yuriw4-testing-20240423.151325-quincy 2024-04-24T17:45:54.523 INFO:teuthology.packaging:sha1: 330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T17:45:54.523 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuriw4-testing-20240423.151325-quincy 2024-04-24T17:45:54.885 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-915.g330067db 2024-04-24T17:45:54.886 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-24T17:45:54.895 INFO:teuthology.task.internal:no buildpackages task found 2024-04-24T17:45:54.895 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-24T17:45:54.907 INFO:teuthology.task.internal:Saving configuration 2024-04-24T17:45:54.920 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-24T17:45:54.930 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-24T17:45:54.953 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi049.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 17:39:12.210201', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAmpKeVOaV+kk9DAKzd4G75t9zj75Y7RbtC5dbT4qkkJnSiDmlEp5teDAKuw9Y2Pahlg5fVaJUzSKw+9SIOVJkg='} 2024-04-24T17:45:54.973 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi078.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'rhel', 'os_version': '8.6', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-24 17:39:12.208811', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfZwGy4KggpIQ5Xr3FpzGaV92di5pOwGC7vllv/74eSJo0Lobhx/D49PoG7Ya/JvdGWv84/cQGpAO67zRImDz4='} 2024-04-24T17:45:54.973 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-24T17:45:54.983 INFO:teuthology.task.internal:roles: ubuntu@smithi049.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-04-24T17:45:54.983 INFO:teuthology.task.internal:roles: ubuntu@smithi078.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-04-24T17:45:54.984 INFO:teuthology.run_tasks:Running task console_log... 2024-04-24T17:45:55.063 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f9ea8975f70>, signals=[15]) 2024-04-24T17:45:55.063 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-24T17:45:55.073 INFO:teuthology.task.internal:Opening connections... 2024-04-24T17:45:55.073 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T17:45:55.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T17:45:55.158 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T17:45:55.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T17:45:55.232 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-24T17:45:55.239 DEBUG:teuthology.orchestra.run.smithi049:> uname -m 2024-04-24T17:45:55.260 INFO:teuthology.orchestra.run.smithi049.stdout:x86_64 2024-04-24T17:45:55.260 DEBUG:teuthology.orchestra.run.smithi049:> cat /etc/os-release 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:NAME="Red Hat Enterprise Linux" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION="8.6 (Ootpa)" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:ID="rhel" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:ID_LIKE="fedora" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:VERSION_ID="8.6" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:PLATFORM_ID="platform:el8" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:ANSI_COLOR="0;31" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-24T17:45:55.320 INFO:teuthology.orchestra.run.smithi049.stdout:HOME_URL="https://www.redhat.com/" 2024-04-24T17:45:55.321 INFO:teuthology.orchestra.run.smithi049.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-24T17:45:55.321 INFO:teuthology.orchestra.run.smithi049.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T17:45:55.321 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:45:55.321 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-24T17:45:55.321 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-24T17:45:55.321 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-24T17:45:55.321 INFO:teuthology.orchestra.run.smithi049.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-24T17:45:55.322 INFO:teuthology.lock.ops:Updating smithi049.front.sepia.ceph.com on lock server 2024-04-24T17:45:55.347 DEBUG:teuthology.orchestra.run.smithi078:> uname -m 2024-04-24T17:45:55.368 INFO:teuthology.orchestra.run.smithi078.stdout:x86_64 2024-04-24T17:45:55.368 DEBUG:teuthology.orchestra.run.smithi078:> cat /etc/os-release 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:NAME="Red Hat Enterprise Linux" 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION="8.6 (Ootpa)" 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:ID="rhel" 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:ID_LIKE="fedora" 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:VERSION_ID="8.6" 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:PLATFORM_ID="platform:el8" 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:PRETTY_NAME="Red Hat Enterprise Linux 8.6 (Ootpa)" 2024-04-24T17:45:55.427 INFO:teuthology.orchestra.run.smithi078.stdout:ANSI_COLOR="0;31" 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:CPE_NAME="cpe:/o:redhat:enterprise_linux:8::baseos" 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:HOME_URL="https://www.redhat.com/" 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:DOCUMENTATION_URL="https://access.redhat.com/documentation/red_hat_enterprise_linux/8/" 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 8" 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_BUGZILLA_PRODUCT_VERSION=8.6 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" 2024-04-24T17:45:55.428 INFO:teuthology.orchestra.run.smithi078.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="8.6" 2024-04-24T17:45:55.428 INFO:teuthology.lock.ops:Updating smithi078.front.sepia.ceph.com on lock server 2024-04-24T17:45:55.447 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-24T17:45:55.457 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-24T17:45:55.466 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-24T17:45:55.466 DEBUG:teuthology.orchestra.run.smithi049:> test '!' -e /home/ubuntu/cephtest 2024-04-24T17:45:55.469 DEBUG:teuthology.orchestra.run.smithi078:> test '!' -e /home/ubuntu/cephtest 2024-04-24T17:45:55.488 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-24T17:45:55.495 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-24T17:45:55.495 DEBUG:teuthology.orchestra.run.smithi049:> test -z $(ls -A /var/lib/ceph) 2024-04-24T17:45:55.528 DEBUG:teuthology.orchestra.run.smithi078:> test -z $(ls -A /var/lib/ceph) 2024-04-24T17:45:55.553 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-24T17:45:55.596 INFO:teuthology.run_tasks:Running task kernel... 2024-04-24T17:45:55.611 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-24T17:45:55.611 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-24T17:45:55.611 DEBUG:teuthology.orchestra.run.smithi049:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T17:45:55.612 DEBUG:teuthology.orchestra.run.smithi078:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-24T17:45:55.631 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T17:45:55.631 DEBUG:teuthology.orchestra.run.smithi078:> uname -r 2024-04-24T17:45:55.632 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T17:45:55.632 DEBUG:teuthology.orchestra.run.smithi049:> uname -r 2024-04-24T17:45:55.692 INFO:teuthology.orchestra.run.smithi078.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:45:55.692 INFO:teuthology.task.kernel:Running kernel on smithi078: 4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:45:55.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum install -y kernel 2024-04-24T17:45:55.693 INFO:teuthology.orchestra.run.smithi049.stdout:4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:45:55.693 INFO:teuthology.task.kernel:Running kernel on smithi049: 4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:45:55.693 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum install -y kernel 2024-04-24T17:45:57.211 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T17:45:57.211 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2024-04-24T17:45:57.222 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:45:57.223 INFO:teuthology.orchestra.run.smithi078.stdout:Unable to read consumer identity 2024-04-24T17:45:57.427 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:45:57.427 INFO:teuthology.orchestra.run.smithi078.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-24T17:45:57.427 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:45:57.484 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:45:57.484 INFO:teuthology.orchestra.run.smithi049.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-24T17:45:57.484 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:45:59.337 INFO:teuthology.orchestra.run.smithi078.stdout:Copr repo for python3-asyncssh owned by ceph 12 kB/s | 3.5 kB 00:00 2024-04-24T17:45:59.450 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-04-24T17:46:01.079 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 9.5 MB/s | 16 MB 00:01 2024-04-24T17:46:03.234 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 4.3 MB/s | 16 MB 00:03 2024-04-24T17:46:04.704 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 299 kB/s | 24 kB 00:00 2024-04-24T17:46:05.448 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-24T17:46:05.449 INFO:teuthology.orchestra.run.smithi078.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-24T17:46:05.536 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T17:46:05.537 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T17:46:05.537 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T17:46:05.667 DEBUG:teuthology.orchestra.run.smithi078:> echo no | sudo yum reinstall kernel || true 2024-04-24T17:46:06.004 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:46:06.004 INFO:teuthology.orchestra.run.smithi078.stdout:Unable to read consumer identity 2024-04-24T17:46:06.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:46:06.160 INFO:teuthology.orchestra.run.smithi078.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-24T17:46:06.160 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:46:06.472 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 05:46:04 PM UTC. 2024-04-24T17:46:06.569 INFO:teuthology.orchestra.run.smithi078.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-24T17:46:06.569 INFO:teuthology.orchestra.run.smithi078.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-24T17:46:06.615 INFO:teuthology.orchestra.run.smithi078.stderr:Error: No packages marked for reinstall. 2024-04-24T17:46:06.662 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum reinstall -y kernel || true 2024-04-24T17:46:06.999 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:46:06.999 INFO:teuthology.orchestra.run.smithi078.stdout:Unable to read consumer identity 2024-04-24T17:46:07.076 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 461 kB/s | 24 kB 00:00 2024-04-24T17:46:07.148 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:46:07.148 INFO:teuthology.orchestra.run.smithi078.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-24T17:46:07.148 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:46:07.452 INFO:teuthology.orchestra.run.smithi078.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 05:46:04 PM UTC. 2024-04-24T17:46:07.519 INFO:teuthology.orchestra.run.smithi078.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-24T17:46:07.520 INFO:teuthology.orchestra.run.smithi078.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-24T17:46:07.562 INFO:teuthology.orchestra.run.smithi078.stderr:Error: No packages marked for reinstall. 2024-04-24T17:46:07.608 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T17:46:07.758 INFO:teuthology.orchestra.run.smithi078.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:46:07.758 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:46:07.758 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T17:46:07.758 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-04-24T17:46:07.758 DEBUG:teuthology.orchestra.run.smithi078:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T17:46:07.793 INFO:teuthology.orchestra.run.smithi078.stdout:ttyS1 2024-04-24T17:46:07.808 DEBUG:teuthology.parallel:result is None 2024-04-24T17:46:07.837 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-4.18.0-513.9.1.el8_9.x86_64 is already installed. 2024-04-24T17:46:07.837 INFO:teuthology.orchestra.run.smithi049.stdout:Package kernel-4.18.0-372.9.1.el8.x86_64 is already installed. 2024-04-24T17:46:07.921 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T17:46:07.923 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T17:46:07.923 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T17:46:08.075 DEBUG:teuthology.orchestra.run.smithi049:> echo no | sudo yum reinstall kernel || true 2024-04-24T17:46:08.430 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T17:46:08.430 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2024-04-24T17:46:08.700 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:46:08.700 INFO:teuthology.orchestra.run.smithi049.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-24T17:46:08.700 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:46:09.028 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:02 ago on Wed 24 Apr 2024 05:46:07 PM UTC. 2024-04-24T17:46:09.115 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-24T17:46:09.115 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-24T17:46:09.181 INFO:teuthology.orchestra.run.smithi049.stderr:Error: No packages marked for reinstall. 2024-04-24T17:46:09.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum reinstall -y kernel || true 2024-04-24T17:46:09.585 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T17:46:09.585 INFO:teuthology.orchestra.run.smithi049.stdout:Unable to read consumer identity 2024-04-24T17:46:09.807 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:46:09.807 INFO:teuthology.orchestra.run.smithi049.stdout:This system is not registered with an entitlement server. You can use subscription-manager to register. 2024-04-24T17:46:09.807 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:46:10.124 INFO:teuthology.orchestra.run.smithi049.stdout:Last metadata expiration check: 0:00:03 ago on Wed 24 Apr 2024 05:46:07 PM UTC. 2024-04-24T17:46:10.194 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-513.9.1.el8_9.x86_64 (from rhel-8-for-x86_64-baseos-rpms) not available. 2024-04-24T17:46:10.194 INFO:teuthology.orchestra.run.smithi049.stdout:Installed package kernel-4.18.0-372.9.1.el8.x86_64 (from anaconda) not available. 2024-04-24T17:46:10.234 INFO:teuthology.orchestra.run.smithi049.stderr:Error: No packages marked for reinstall. 2024-04-24T17:46:10.288 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q kernel | sort -rV | head -n 1 2024-04-24T17:46:10.466 INFO:teuthology.orchestra.run.smithi049.stdout:kernel-4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:46:10.466 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-513.9.1.el8_9.x86_64 2024-04-24T17:46:10.467 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-24T17:46:10.467 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-24T17:46:10.467 DEBUG:teuthology.orchestra.run.smithi049:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-24T17:46:10.502 INFO:teuthology.orchestra.run.smithi049.stdout:ttyS1 2024-04-24T17:46:10.520 DEBUG:teuthology.parallel:result is None 2024-04-24T17:46:10.520 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-24T17:46:10.532 INFO:teuthology.task.internal:Creating test directory... 2024-04-24T17:46:10.532 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T17:46:10.563 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-24T17:46:10.582 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-24T17:46:10.593 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-24T17:46:10.607 INFO:teuthology.task.internal:Creating archive directory... 2024-04-24T17:46:10.607 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T17:46:10.620 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-24T17:46:10.658 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-24T17:46:10.666 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-24T17:46:10.666 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T17:46:10.689 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-24T17:46:10.726 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T17:46:10.734 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T17:46:10.737 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T17:46:10.747 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-24T17:46:10.748 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-24T17:46:10.755 INFO:teuthology.task.internal:Configuring sudo... 2024-04-24T17:46:10.755 DEBUG:teuthology.orchestra.run.smithi049:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T17:46:10.781 DEBUG:teuthology.orchestra.run.smithi078:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-24T17:46:10.822 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-24T17:46:10.832 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-24T17:46:10.832 DEBUG:teuthology.orchestra.run.smithi049:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T17:46:10.849 DEBUG:teuthology.orchestra.run.smithi078:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-24T17:46:10.883 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T17:46:10.948 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T17:46:10.996 DEBUG:teuthology.orchestra.run.smithi049:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T17:46:11.079 DEBUG:teuthology.orchestra.run.smithi049:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T17:46:11.122 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T17:46:11.122 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T17:46:11.189 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T17:46:11.235 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-24T17:46:11.286 DEBUG:teuthology.orchestra.run.smithi078:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T17:46:11.371 DEBUG:teuthology.orchestra.run.smithi078:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-24T17:46:11.416 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T17:46:11.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-24T17:46:11.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo service rsyslog restart 2024-04-24T17:46:11.486 DEBUG:teuthology.orchestra.run.smithi078:> sudo service rsyslog restart 2024-04-24T17:46:11.529 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T17:46:11.568 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T17:46:12.005 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-24T17:46:12.015 INFO:teuthology.task.internal:Starting timer... 2024-04-24T17:46:12.015 INFO:teuthology.run_tasks:Running task pcp... 2024-04-24T17:46:12.031 INFO:teuthology.run_tasks:Running task selinux... 2024-04-24T17:46:12.040 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-04-24T17:46:12.041 DEBUG:teuthology.orchestra.run.smithi049:> sudo service auditd rotate 2024-04-24T17:46:12.124 INFO:teuthology.orchestra.run.smithi049.stdout:Rotating logs: 2024-04-24T17:46:12.125 DEBUG:teuthology.orchestra.run.smithi078:> sudo service auditd rotate 2024-04-24T17:46:12.197 INFO:teuthology.orchestra.run.smithi078.stdout:Rotating logs: 2024-04-24T17:46:12.199 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-24T17:46:12.199 DEBUG:teuthology.orchestra.run.smithi049:> /usr/sbin/getenforce 2024-04-24T17:46:12.222 INFO:teuthology.orchestra.run.smithi049.stdout:Permissive 2024-04-24T17:46:12.222 DEBUG:teuthology.orchestra.run.smithi078:> /usr/sbin/getenforce 2024-04-24T17:46:12.243 INFO:teuthology.orchestra.run.smithi078.stdout:Permissive 2024-04-24T17:46:12.243 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi049.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi078.front.sepia.ceph.com': 'permissive'} 2024-04-24T17:46:12.243 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T17:46:12.291 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T17:46:12.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T17:46:12.319 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T17:46:12.320 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-24T17:46:12.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo /usr/sbin/setenforce permissive 2024-04-24T17:46:12.360 DEBUG:teuthology.orchestra.run.smithi078:> sudo /usr/sbin/setenforce permissive 2024-04-24T17:46:12.390 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-24T17:46:12.401 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-24T17:46:12.418 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-24T17:46:12.419 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi049.front.sepia.ceph.com,smithi078.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-24T17:56:57.985 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi049.front.sepia.ceph.com'), Remote(name='ubuntu@smithi078.front.sepia.ceph.com')] 2024-04-24T17:56:57.986 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-24T17:56:57.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi049.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T17:56:58.071 DEBUG:teuthology.orchestra.run.smithi049:> true 2024-04-24T17:56:58.140 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi049.front.sepia.ceph.com' 2024-04-24T17:56:58.141 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-24T17:56:58.141 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi078.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-24T17:56:58.217 DEBUG:teuthology.orchestra.run.smithi078:> true 2024-04-24T17:56:58.285 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi078.front.sepia.ceph.com' 2024-04-24T17:56:58.285 INFO:teuthology.run_tasks:Running task clock... 2024-04-24T17:56:58.296 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-24T17:56:58.296 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T17:56:58.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T17:56:58.299 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-24T17:56:58.299 DEBUG:teuthology.orchestra.run.smithi078:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T17:56:58.331 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T17:56:58.347 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T17:56:58.357 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-24T17:56:58.371 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-24T17:56:58.416 INFO:teuthology.orchestra.run.smithi049.stderr:sudo: ntpd: command not found 2024-04-24T17:56:58.428 INFO:teuthology.orchestra.run.smithi049.stdout:506 Cannot talk to daemon 2024-04-24T17:56:58.438 INFO:teuthology.orchestra.run.smithi078.stderr:sudo: ntpd: command not found 2024-04-24T17:56:58.443 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T17:56:58.450 INFO:teuthology.orchestra.run.smithi078.stdout:506 Cannot talk to daemon 2024-04-24T17:56:58.458 INFO:teuthology.orchestra.run.smithi049.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T17:56:58.465 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-24T17:56:58.479 INFO:teuthology.orchestra.run.smithi078.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-24T17:56:58.540 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2024-04-24T17:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T17:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-04-24T17:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.543 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.544 INFO:teuthology.orchestra.run.smithi078.stderr:bash: ntpq: command not found 2024-04-24T17:56:58.547 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T17:56:58.547 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-24T17:56:58.547 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.547 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.547 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.547 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T17:56:58.548 INFO:teuthology.run_tasks:Running task install... 2024-04-24T17:56:58.556 DEBUG:teuthology.task.install:project ceph 2024-04-24T17:56:58.556 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}} 2024-04-24T17:56:58.557 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'} 2024-04-24T17:56:58.557 INFO:teuthology.task.install:Using flavor: default 2024-04-24T17:56:58.562 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-24T17:56:58.563 INFO:teuthology.task.install:extra packages: [] 2024-04-24T17:56:58.563 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'tag': None, 'wait_for_package': False} 2024-04-24T17:56:58.563 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T17:56:58.565 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'tag': None, 'wait_for_package': False} 2024-04-24T17:56:58.566 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T17:56:58.722 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/ 2024-04-24T17:56:58.722 INFO:teuthology.task.install.rpm:Package version is 17.2.7-915.g330067db 2024-04-24T17:56:58.788 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/ 2024-04-24T17:56:58.788 INFO:teuthology.task.install.rpm:Package version is 17.2.7-915.g330067db 2024-04-24T17:56:58.976 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T17:56:58.976 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T17:56:58.977 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T17:56:59.006 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T17:56:59.006 DEBUG:teuthology.orchestra.run.smithi049:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/330067dbce47655d1f762ef14d546d2fde16a6a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T17:56:59.045 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-yuriw4-testing-20240423.151325-quincy/330067dbce47655d1f762ef14d546d2fde16a6a9/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-24T17:56:59.045 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T17:56:59.045 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-24T17:56:59.076 DEBUG:teuthology.orchestra.run.smithi049:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T17:56:59.077 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-24T17:56:59.077 DEBUG:teuthology.orchestra.run.smithi078:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/330067dbce47655d1f762ef14d546d2fde16a6a9/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-24T17:56:59.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-24T17:56:59.155 DEBUG:teuthology.orchestra.run.smithi049:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T17:56:59.222 INFO:teuthology.orchestra.run.smithi049.stdout:check_obsoletes = 1 2024-04-24T17:56:59.224 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum clean all 2024-04-24T17:56:59.227 DEBUG:teuthology.orchestra.run.smithi078:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-24T17:56:59.293 INFO:teuthology.orchestra.run.smithi078.stdout:check_obsoletes = 1 2024-04-24T17:56:59.295 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum clean all 2024-04-24T17:56:59.618 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T17:56:59.674 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:57:00.968 INFO:teuthology.orchestra.run.smithi078.stdout:63 files removed 2024-04-24T17:57:00.987 INFO:teuthology.orchestra.run.smithi049.stdout:63 files removed 2024-04-24T17:57:01.021 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-radosgw 2024-04-24T17:57:01.040 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-radosgw 2024-04-24T17:57:01.357 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:57:01.399 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T17:57:03.183 INFO:teuthology.orchestra.run.smithi049.stdout:ceph packages for x86_64 166 kB/s | 78 kB 00:00 2024-04-24T17:57:03.475 INFO:teuthology.orchestra.run.smithi078.stdout:ceph packages for x86_64 141 kB/s | 78 kB 00:00 2024-04-24T17:57:03.667 INFO:teuthology.orchestra.run.smithi049.stdout:ceph noarch packages 24 kB/s | 11 kB 00:00 2024-04-24T17:57:04.075 INFO:teuthology.orchestra.run.smithi078.stdout:ceph noarch packages 19 kB/s | 11 kB 00:00 2024-04-24T17:57:04.576 INFO:teuthology.orchestra.run.smithi078.stdout:ceph source packages 3.1 kB/s | 1.4 kB 00:00 2024-04-24T17:57:04.802 INFO:teuthology.orchestra.run.smithi078.stdout:Copr repo for python3-asyncssh owned by ceph 17 kB/s | 3.5 kB 00:00 2024-04-24T17:57:07.226 INFO:teuthology.orchestra.run.smithi049.stdout:ceph source packages 420 B/s | 1.4 kB 00:03 2024-04-24T17:57:07.493 INFO:teuthology.orchestra.run.smithi049.stdout:Copr repo for python3-asyncssh owned by ceph 14 kB/s | 3.5 kB 00:00 2024-04-24T17:57:07.669 INFO:teuthology.orchestra.run.smithi078.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 16 MB 00:02 2024-04-24T17:57:11.295 INFO:teuthology.orchestra.run.smithi078.stdout:lab-extras 393 kB/s | 24 kB 00:00 2024-04-24T17:57:13.046 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 MB/s | 60 MB 00:01 2024-04-24T17:57:20.345 INFO:teuthology.orchestra.run.smithi049.stdout:Extra Packages for Enterprise Linux 1.3 MB/s | 16 MB 00:12 2024-04-24T17:57:23.913 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 15 MB/s | 8.8 MB 00:00 2024-04-24T17:57:24.215 INFO:teuthology.orchestra.run.smithi049.stdout:lab-extras 294 kB/s | 24 kB 00:00 2024-04-24T17:57:25.727 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 42 MB/s | 60 MB 00:01 2024-04-24T17:57:26.939 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 51 MB/s | 67 MB 00:01 2024-04-24T17:57:37.452 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat CodeReady Linux Builder for RHEL 8 x86_ 13 MB/s | 8.8 MB 00:00 2024-04-24T17:57:40.646 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 46 MB/s | 67 MB 00:01 2024-04-24T17:57:42.197 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout:=================================================================================================== 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout:=================================================================================================== 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrading: 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: librados2 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.6 M 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base x86_64 2:17.2.7-915.g330067db.el8 ceph 5.3 M 2024-04-24T17:57:42.200 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 24 M 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux x86_64 2:17.2.7-915.g330067db.el8 ceph 24 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2 x86_64 2:17.2.7-915.g330067db.el8 ceph 798 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1 x86_64 2:17.2.7-915.g330067db.el8 ceph 505 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2 x86_64 2:17.2.7-915.g330067db.el8 ceph 5.6 M 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse x86_64 2:17.2.7-915.g330067db.el8 ceph 45 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 117 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs x86_64 2:17.2.7-915.g330067db.el8 ceph 190 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados x86_64 2:17.2.7-915.g330067db.el8 ceph 370 k 2024-04-24T17:57:42.201 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd x86_64 2:17.2.7-915.g330067db.el8 ceph 358 k 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw x86_64 2:17.2.7-915.g330067db.el8 ceph 107 k 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout:=================================================================================================== 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout:Install 21 Packages 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout:Upgrade 2 Packages 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:57:42.202 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 64 M 2024-04-24T17:57:42.203 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T17:57:45.064 INFO:teuthology.orchestra.run.smithi078.stdout:(1/23): ceph-radosgw-17.2.7-915.g330067db.el8.x 6.2 MB/s | 17 MB 00:02 2024-04-24T17:57:45.131 INFO:teuthology.orchestra.run.smithi078.stdout:(2/23): ceph-selinux-17.2.7-915.g330067db.el8.x 366 kB/s | 24 kB 00:00 2024-04-24T17:57:45.281 INFO:teuthology.orchestra.run.smithi078.stdout:(3/23): libcephfs2-17.2.7-915.g330067db.el8.x86 5.2 MB/s | 798 kB 00:00 2024-04-24T17:57:45.373 INFO:teuthology.orchestra.run.smithi078.stdout:(4/23): libradosstriper1-17.2.7-915.g330067db.e 5.4 MB/s | 505 kB 00:00 2024-04-24T17:57:45.566 INFO:teuthology.orchestra.run.smithi078.stdout:(5/23): ceph-base-17.2.7-915.g330067db.el8.x86_ 1.6 MB/s | 5.3 MB 00:03 2024-04-24T17:57:45.941 INFO:teuthology.orchestra.run.smithi078.stdout:(6/23): ceph-common-17.2.7-915.g330067db.el8.x8 6.6 MB/s | 24 MB 00:03 2024-04-24T17:57:45.966 INFO:teuthology.orchestra.run.smithi078.stdout:(7/23): python3-ceph-argparse-17.2.7-915.g33006 112 kB/s | 45 kB 00:00 2024-04-24T17:57:45.991 INFO:teuthology.orchestra.run.smithi078.stdout:(8/23): python3-ceph-common-17.2.7-915.g330067d 2.3 MB/s | 117 kB 00:00 2024-04-24T17:57:46.100 INFO:teuthology.orchestra.run.smithi078.stdout:(9/23): librgw2-17.2.7-915.g330067db.el8.x86_64 7.7 MB/s | 5.6 MB 00:00 2024-04-24T17:57:46.117 INFO:teuthology.orchestra.run.smithi078.stdout:(10/23): python3-cephfs-17.2.7-915.g330067db.el 1.2 MB/s | 190 kB 00:00 2024-04-24T17:57:46.142 INFO:teuthology.orchestra.run.smithi078.stdout:(11/23): python3-rados-17.2.7-915.g330067db.el8 2.4 MB/s | 370 kB 00:00 2024-04-24T17:57:46.167 INFO:teuthology.orchestra.run.smithi078.stdout:(12/23): python3-rbd-17.2.7-915.g330067db.el8.x 5.2 MB/s | 358 kB 00:00 2024-04-24T17:57:46.210 INFO:teuthology.orchestra.run.smithi078.stdout:(13/23): python3-rgw-17.2.7-915.g330067db.el8.x 1.1 MB/s | 107 kB 00:00 2024-04-24T17:57:46.485 INFO:teuthology.orchestra.run.smithi078.stdout:(14/23): liboath-2.6.2-3.el8.x86_64.rpm 185 kB/s | 59 kB 00:00 2024-04-24T17:57:46.668 INFO:teuthology.orchestra.run.smithi078.stdout:(15/23): gperftools-libs-2.7-9.el8.x86_64.rpm 583 kB/s | 306 kB 00:00 2024-04-24T17:57:46.685 INFO:teuthology.orchestra.run.smithi078.stdout:(16/23): libunwind-1.3.1-3.el8.x86_64.rpm 160 kB/s | 75 kB 00:00 2024-04-24T17:57:46.877 INFO:teuthology.orchestra.run.smithi078.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.3 MB/s | 259 kB 00:00 2024-04-24T17:57:46.935 INFO:teuthology.orchestra.run.smithi078.stdout:(18/23): python3-prettytable-0.7.2-14.el8.noarc 165 kB/s | 44 kB 00:00 2024-04-24T17:57:47.003 INFO:teuthology.orchestra.run.smithi078.stdout:(19/23): librdkafka-0.11.4-3.el8.x86_64.rpm 2.8 MB/s | 354 kB 00:00 2024-04-24T17:57:47.044 INFO:teuthology.orchestra.run.smithi078.stdout:(20/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 434 kB/s | 47 kB 00:00 2024-04-24T17:57:47.120 INFO:teuthology.orchestra.run.smithi078.stdout:(21/23): thrift-0.13.0-2.el8.x86_64.rpm 2.7 MB/s | 1.7 MB 00:00 2024-04-24T17:57:47.445 INFO:teuthology.orchestra.run.smithi078.stdout:(22/23): librbd1-17.2.7-915.g330067db.el8.x86_6 9.0 MB/s | 3.6 MB 00:00 2024-04-24T17:57:47.762 INFO:teuthology.orchestra.run.smithi078.stdout:(23/23): librados2-17.2.7-915.g330067db.el8.x86 4.6 MB/s | 3.5 MB 00:00 2024-04-24T17:57:47.766 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T17:57:47.766 INFO:teuthology.orchestra.run.smithi078.stdout:Total 11 MB/s | 64 MB 00:05 2024-04-24T17:57:47.814 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T17:57:47.859 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T17:57:47.859 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T17:57:48.204 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T17:57:48.213 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T17:57:49.318 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T17:57:49.475 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2024-04-24T17:57:49.859 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2024-04-24T17:57:50.059 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librados2-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T17:57:50.160 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T17:57:50.330 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T17:57:50.591 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2024-04-24T17:57:50.791 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T17:57:51.170 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T17:57:51.642 INFO:teuthology.orchestra.run.smithi078.stdout: Upgrading : librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T17:57:51.709 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T17:57:51.940 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/25 2024-04-24T17:57:52.117 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-24T17:57:52.335 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-24T17:57:53.004 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2024-04-24T17:57:53.183 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T17:57:53.289 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T17:57:53.476 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 11/25 2024-04-24T17:57:53.669 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T17:57:53.934 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T17:57:54.177 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 14/25 2024-04-24T17:57:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T17:57:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T17:57:54.535 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-24T17:57:54.774 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2024-04-24T17:57:55.033 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-24T17:57:55.183 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 19/25 2024-04-24T17:57:57.369 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout:=================================================================================================== 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout:=================================================================================================== 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrading: 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout: librados2 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1 x86_64 2:17.2.7-915.g330067db.el8 ceph 3.6 M 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T17:57:57.372 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base x86_64 2:17.2.7-915.g330067db.el8 ceph 5.3 M 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 24 M 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux x86_64 2:17.2.7-915.g330067db.el8 ceph 24 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2 x86_64 2:17.2.7-915.g330067db.el8 ceph 798 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq x86_64 0.9.0-5.el8_9 rhel-8-for-x86_64-baseos-rpms 47 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1 x86_64 2:17.2.7-915.g330067db.el8 ceph 505 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka x86_64 0.11.4-3.el8 rhel-8-for-x86_64-appstream-rpms 354 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2 x86_64 2:17.2.7-915.g330067db.el8 ceph 5.6 M 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust x86_64 2.8.1-11.el8 rhel-8-for-x86_64-appstream-rpms 259 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse x86_64 2:17.2.7-915.g330067db.el8 ceph 45 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common x86_64 2:17.2.7-915.g330067db.el8 ceph 117 k 2024-04-24T17:57:57.373 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs x86_64 2:17.2.7-915.g330067db.el8 ceph 190 k 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable noarch 0.7.2-14.el8 rhel-8-for-x86_64-appstream-rpms 44 k 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados x86_64 2:17.2.7-915.g330067db.el8 ceph 370 k 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd x86_64 2:17.2.7-915.g330067db.el8 ceph 358 k 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw x86_64 2:17.2.7-915.g330067db.el8 ceph 107 k 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:=================================================================================================== 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Install 21 Packages 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout:Upgrade 2 Packages 2024-04-24T17:57:57.374 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:57:57.375 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 64 M 2024-04-24T17:57:57.375 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T17:57:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T17:57:59.177 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T17:57:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T17:58:00.004 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T17:58:00.092 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T17:58:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T17:58:02.861 INFO:teuthology.orchestra.run.smithi049.stdout:(1/23): ceph-base-17.2.7-915.g330067db.el8.x86_ 1.0 MB/s | 5.3 MB 00:05 2024-04-24T17:58:02.952 INFO:teuthology.orchestra.run.smithi049.stdout:(2/23): ceph-selinux-17.2.7-915.g330067db.el8.x 266 kB/s | 24 kB 00:00 2024-04-24T17:58:03.580 INFO:teuthology.orchestra.run.smithi049.stdout:(3/23): libcephfs2-17.2.7-915.g330067db.el8.x86 1.2 MB/s | 798 kB 00:00 2024-04-24T17:58:04.061 INFO:teuthology.orchestra.run.smithi049.stdout:(4/23): libradosstriper1-17.2.7-915.g330067db.e 1.0 MB/s | 505 kB 00:00 2024-04-24T17:58:07.478 INFO:teuthology.orchestra.run.smithi049.stdout:(5/23): librgw2-17.2.7-915.g330067db.el8.x86_64 1.6 MB/s | 5.6 MB 00:03 2024-04-24T17:58:07.578 INFO:teuthology.orchestra.run.smithi049.stdout:(6/23): python3-ceph-argparse-17.2.7-915.g33006 448 kB/s | 45 kB 00:00 2024-04-24T17:58:07.678 INFO:teuthology.orchestra.run.smithi049.stdout:(7/23): python3-ceph-common-17.2.7-915.g330067d 1.1 MB/s | 117 kB 00:00 2024-04-24T17:58:07.884 INFO:teuthology.orchestra.run.smithi049.stdout:(8/23): python3-cephfs-17.2.7-915.g330067db.el8 923 kB/s | 190 kB 00:00 2024-04-24T17:58:08.237 INFO:teuthology.orchestra.run.smithi049.stdout:(9/23): python3-rados-17.2.7-915.g330067db.el8. 1.0 MB/s | 370 kB 00:00 2024-04-24T17:58:08.554 INFO:teuthology.orchestra.run.smithi049.stdout:(10/23): python3-rbd-17.2.7-915.g330067db.el8.x 1.1 MB/s | 358 kB 00:00 2024-04-24T17:58:08.737 INFO:teuthology.orchestra.run.smithi049.stdout:(11/23): python3-rgw-17.2.7-915.g330067db.el8.x 581 kB/s | 107 kB 00:00 2024-04-24T17:58:08.871 INFO:teuthology.orchestra.run.smithi049.stdout:(12/23): gperftools-libs-2.7-9.el8.x86_64.rpm 2.2 MB/s | 306 kB 00:00 2024-04-24T17:58:08.904 INFO:teuthology.orchestra.run.smithi049.stdout:(13/23): liboath-2.6.2-3.el8.x86_64.rpm 1.7 MB/s | 59 kB 00:00 2024-04-24T17:58:08.947 INFO:teuthology.orchestra.run.smithi049.stdout:(14/23): libunwind-1.3.1-3.el8.x86_64.rpm 1.8 MB/s | 75 kB 00:00 2024-04-24T17:58:09.080 INFO:teuthology.orchestra.run.smithi049.stdout:(15/23): thrift-0.13.0-2.el8.x86_64.rpm 13 MB/s | 1.7 MB 00:00 2024-04-24T17:58:09.280 INFO:teuthology.orchestra.run.smithi049.stdout:(16/23): python3-prettytable-0.7.2-14.el8.noarc 220 kB/s | 44 kB 00:00 2024-04-24T17:58:09.431 INFO:teuthology.orchestra.run.smithi049.stdout:(17/23): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.7 MB/s | 259 kB 00:00 2024-04-24T17:58:09.598 INFO:teuthology.orchestra.run.smithi049.stdout:(18/23): librdkafka-0.11.4-3.el8.x86_64.rpm 2.1 MB/s | 354 kB 00:00 2024-04-24T17:58:09.698 INFO:teuthology.orchestra.run.smithi049.stdout:(19/23): librabbitmq-0.9.0-5.el8_9.x86_64.rpm 471 kB/s | 47 kB 00:00 2024-04-24T17:58:12.765 INFO:teuthology.orchestra.run.smithi049.stdout:(20/23): librados2-17.2.7-915.g330067db.el8.x86 1.1 MB/s | 3.5 MB 00:03 2024-04-24T17:58:12.990 INFO:teuthology.orchestra.run.smithi049.stdout:(21/23): ceph-radosgw-17.2.7-915.g330067db.el8. 1.1 MB/s | 17 MB 00:15 2024-04-24T17:58:15.857 INFO:teuthology.orchestra.run.smithi049.stdout:(22/23): librbd1-17.2.7-915.g330067db.el8.x86_6 1.2 MB/s | 3.6 MB 00:03 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /sys 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /proc 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /dev 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /run 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /mnt 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /var/tmp 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /home 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /tmp 2024-04-24T17:58:19.378 INFO:teuthology.orchestra.run.smithi078.stdout:skipping the directory /dev 2024-04-24T17:58:19.379 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:58:21.043 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T17:58:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T17:58:21.117 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T17:58:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T17:58:21.332 INFO:teuthology.orchestra.run.smithi078.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 1/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 4/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T17:58:22.015 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librgw2-2:17.2.7-915.g330067db.el8.x86_64 7/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 8/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 9/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 11/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 20/25 2024-04-24T17:58:22.016 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 21/25 2024-04-24T17:58:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T17:58:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-24T17:58:22.017 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-2:17.2.7-915.g330067db.el8.x86_64 24/25 2024-04-24T17:58:22.764 INFO:teuthology.orchestra.run.smithi049.stdout:(23/23): ceph-common-17.2.7-915.g330067db.el8.x 958 kB/s | 24 MB 00:25 2024-04-24T17:58:22.767 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T17:58:22.767 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.5 MB/s | 64 MB 00:25 2024-04-24T17:58:22.817 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T17:58:22.866 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T17:58:22.867 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T17:58:23.235 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T17:58:23.245 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T17:58:24.408 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T17:58:24.567 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 1/25 2024-04-24T17:58:24.966 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 1/25 2024-04-24T17:58:25.209 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librados2-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T17:58:25.431 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T17:58:25.598 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T17:58:25.865 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : liboath-2.6.2-3.el8.x86_64 4/25 2024-04-24T17:58:26.042 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T17:58:26.490 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T17:58:26.676 INFO:teuthology.orchestra.run.smithi049.stdout: Upgrading : librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T17:58:26.771 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T17:58:26.967 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librabbitmq-0.9.0-5.el8_9.x86_64 7/25 2024-04-24T17:58:27.134 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-24T17:58:27.365 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 8/25 2024-04-24T17:58:28.080 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : thrift-0.13.0-2.el8.x86_64 9/25 2024-04-24T17:58:28.242 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T17:58:28.310 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librgw2-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T17:58:28.495 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 11/25 2024-04-24T17:58:28.670 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T17:58:28.957 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T17:58:29.231 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 14/25 2024-04-24T17:58:29.425 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T17:58:29.482 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 15/25 2024-04-24T17:58:29.615 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 16/25 2024-04-24T17:58:29.845 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 17/25 2024-04-24T17:58:29.984 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T17:58:29.985 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T17:58:30.158 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 18/25 2024-04-24T17:58:30.317 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 19/25 2024-04-24T17:58:30.528 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:58:30.528 INFO:teuthology.orchestra.run.smithi078.stdout:Upgraded: 2024-04-24T17:58:30.528 INFO:teuthology.orchestra.run.smithi078.stdout: librados2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.528 INFO:teuthology.orchestra.run.smithi078.stdout: librbd1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.528 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T17:58:30.528 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: librgw2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-24T17:58:30.529 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: python3-ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rados-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:58:30.530 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T17:58:30.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-test 2024-04-24T17:58:31.097 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:58:32.812 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 35 kB/s | 4.5 kB 00:00 2024-04-24T17:58:33.515 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-24T17:58:34.513 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T17:58:34.733 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T17:58:35.339 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 20/25 2024-04-24T17:58:35.628 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T17:58:35.697 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 21/25 2024-04-24T17:58:35.802 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T17:58:35.803 INFO:teuthology.orchestra.run.smithi078.stdout:=========================================================================================== 2024-04-24T17:58:35.803 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout:=========================================================================================== 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test x86_64 2:17.2.7-915.g330067db.el8 ceph 53 M 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite x86_64 2:17.2.7-915.g330067db.el8 ceph 179 k 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout:=========================================================================================== 2024-04-24T17:58:35.804 INFO:teuthology.orchestra.run.smithi078.stdout:Install 6 Packages 2024-04-24T17:58:35.805 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:58:35.805 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 54 M 2024-04-24T17:58:35.805 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 236 M 2024-04-24T17:58:35.805 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T17:58:35.842 INFO:teuthology.orchestra.run.smithi078.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.9 MB/s | 70 kB 00:00 2024-04-24T17:58:35.849 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T17:58:35.993 INFO:teuthology.orchestra.run.smithi078.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 2.2 MB/s | 323 kB 00:00 2024-04-24T17:58:36.160 INFO:teuthology.orchestra.run.smithi078.stdout:(3/6): jq-1.6-7.el8.x86_64.rpm 1.2 MB/s | 202 kB 00:00 2024-04-24T17:58:36.271 INFO:teuthology.orchestra.run.smithi078.stdout:(4/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2024-04-24T17:58:36.293 INFO:teuthology.orchestra.run.smithi078.stdout:(5/6): libcephsqlite-17.2.7-915.g330067db.el8.x 368 kB/s | 179 kB 00:00 2024-04-24T17:58:38.727 INFO:teuthology.orchestra.run.smithi078.stdout:(6/6): ceph-test-17.2.7-915.g330067db.el8.x86_6 18 MB/s | 53 MB 00:02 2024-04-24T17:58:38.729 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T17:58:38.729 INFO:teuthology.orchestra.run.smithi078.stdout:Total 18 MB/s | 54 MB 00:02 2024-04-24T17:58:38.767 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T17:58:38.787 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T17:58:38.788 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T17:58:39.078 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T17:58:39.087 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T17:58:39.592 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T17:58:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-24T17:58:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-24T17:58:40.030 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-24T17:58:40.219 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-24T17:58:40.383 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-24T17:58:40.527 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T17:58:45.547 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T17:58:45.766 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T17:58:46.017 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T17:58:46.017 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 1/6 2024-04-24T17:58:46.018 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2/6 2024-04-24T17:58:46.018 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-24T17:58:46.018 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-24T17:58:46.018 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-24T17:58:55.836 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /sys 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /proc 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /run 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /mnt 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /var/tmp 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /home 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /tmp 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout:skipping the directory /dev 2024-04-24T17:58:55.837 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:58:57.624 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T17:58:57.687 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 23/25 2024-04-24T17:58:57.687 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T17:58:57.906 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 24/25 2024-04-24T17:58:57.907 INFO:teuthology.orchestra.run.smithi049.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T17:58:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T17:58:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-base-2:17.2.7-915.g330067db.el8.x86_64 1/25 2024-04-24T17:58:58.574 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 3/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 4/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 5/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 6/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librgw2-2:17.2.7-915.g330067db.el8.x86_64 7/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x 8/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-ceph-common-2:17.2.7-915.g330067db.el8.x86 9/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 10/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rados-2:17.2.7-915.g330067db.el8.x86_64 11/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 12/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 13/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 14/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 15/25 2024-04-24T17:58:58.575 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 16/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 17/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 18/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 19/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 20/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librabbitmq-0.9.0-5.el8_9.x86_64 21/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-2:17.2.7-915.g330067db.el8.x86_64 22/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 23/25 2024-04-24T17:58:58.576 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-2:17.2.7-915.g330067db.el8.x86_64 24/25 2024-04-24T17:58:59.769 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-24T17:58:59.769 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T17:59:00.304 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: jq-1.6-7.el8.x86_64 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:00.305 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T17:59:00.492 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph 2024-04-24T17:59:00.830 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:59:02.624 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-04-24T17:59:03.327 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-24T17:59:05.646 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T17:59:05.649 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================================== 2024-04-24T17:59:05.649 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T17:59:05.649 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================================== 2024-04-24T17:59:05.649 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: ceph x86_64 2:17.2.7-915.g330067db.el8 ceph 6.4 k 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds x86_64 2:17.2.7-915.g330067db.el8 ceph 2.3 M 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr x86_64 2:17.2.7-915.g330067db.el8 ceph 1.6 M 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 240 k 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon x86_64 2:17.2.7-915.g330067db.el8 ceph 4.0 M 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-24T17:59:05.650 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-24T17:59:05.651 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout:====================================================================================================== 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout:Install 32 Packages 2024-04-24T17:59:05.652 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:05.653 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 29 M 2024-04-24T17:59:05.653 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 113 M 2024-04-24T17:59:05.653 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T17:59:06.294 INFO:teuthology.orchestra.run.smithi078.stdout:(1/32): ceph-17.2.7-915.g330067db.el8.x86_64.rp 21 kB/s | 6.4 kB 00:00 2024-04-24T17:59:08.211 INFO:teuthology.orchestra.run.smithi078.stdout:(2/32): ceph-mgr-17.2.7-915.g330067db.el8.x86_6 726 kB/s | 1.6 MB 00:02 2024-04-24T17:59:08.336 INFO:teuthology.orchestra.run.smithi078.stdout:(3/32): ceph-mds-17.2.7-915.g330067db.el8.x86_6 1.0 MB/s | 2.3 MB 00:02 2024-04-24T17:59:08.570 INFO:teuthology.orchestra.run.smithi078.stdout:(4/32): ceph-mgr-modules-core-17.2.7-915.g33006 1.0 MB/s | 240 kB 00:00 2024-04-24T17:59:08.628 INFO:teuthology.orchestra.run.smithi078.stdout:(5/32): ceph-mon-17.2.7-915.g330067db.el8.x86_6 1.7 MB/s | 4.0 MB 00:02 2024-04-24T17:59:08.912 INFO:teuthology.orchestra.run.smithi078.stdout:(6/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 128 kB/s | 44 kB 00:00 2024-04-24T17:59:09.120 INFO:teuthology.orchestra.run.smithi078.stdout:(7/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 376 kB/s | 185 kB 00:00 2024-04-24T17:59:09.162 INFO:teuthology.orchestra.run.smithi078.stdout:(8/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 689 kB/s | 173 kB 00:00 2024-04-24T17:59:09.263 INFO:teuthology.orchestra.run.smithi078.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 111 kB/s | 11 kB 00:00 2024-04-24T17:59:09.363 INFO:teuthology.orchestra.run.smithi078.stdout:(10/32): python3-cherrypy-18.4.0-1.el8.noarch.r 1.6 MB/s | 384 kB 00:00 2024-04-24T17:59:09.380 INFO:teuthology.orchestra.run.smithi078.stdout:(11/32): python3-jaraco-functools-2.0-4.el8.noa 150 kB/s | 18 kB 00:00 2024-04-24T17:59:09.489 INFO:teuthology.orchestra.run.smithi078.stdout:(12/32): python3-more-itertools-7.2.0-3.el8.noa 541 kB/s | 59 kB 00:00 2024-04-24T17:59:09.506 INFO:teuthology.orchestra.run.smithi078.stdout:(13/32): python3-logutils-0.3.5-11.el8.noarch.r 388 kB/s | 49 kB 00:00 2024-04-24T17:59:09.589 INFO:teuthology.orchestra.run.smithi078.stdout:(14/32): python3-portend-2.6-1.el8.noarch.rpm 187 kB/s | 16 kB 00:00 2024-04-24T17:59:09.623 INFO:teuthology.orchestra.run.smithi078.stdout:(15/32): python3-pecan-1.3.2-9.el8.noarch.rpm 2.1 MB/s | 283 kB 00:00 2024-04-24T17:59:09.698 INFO:teuthology.orchestra.run.smithi078.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 172 kB/s | 19 kB 00:00 2024-04-24T17:59:09.807 INFO:teuthology.orchestra.run.smithi078.stdout:(17/32): python3-tempora-1.14.1-5.el8.noarch.rp 270 kB/s | 29 kB 00:00 2024-04-24T17:59:09.907 INFO:teuthology.orchestra.run.smithi078.stdout:(18/32): python3-singledispatch-3.4.0.3-18.el8. 84 kB/s | 24 kB 00:00 2024-04-24T17:59:09.924 INFO:teuthology.orchestra.run.smithi078.stdout:(19/32): python3-trustme-0.6.0-4.el8.noarch.rpm 236 kB/s | 27 kB 00:00 2024-04-24T17:59:10.049 INFO:teuthology.orchestra.run.smithi078.stdout:(20/32): python3-waitress-1.4.3-1.el8.noarch.rp 1.8 MB/s | 255 kB 00:00 2024-04-24T17:59:10.217 INFO:teuthology.orchestra.run.smithi078.stdout:(21/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 859 kB/s | 251 kB 00:00 2024-04-24T17:59:10.250 INFO:teuthology.orchestra.run.smithi078.stdout:(22/32): python3-webtest-2.0.33-1.el8.noarch.rp 426 kB/s | 85 kB 00:00 2024-04-24T17:59:10.392 INFO:teuthology.orchestra.run.smithi078.stdout:(23/32): python3-werkzeug-0.12.2-4.el8.noarch.r 3.2 MB/s | 457 kB 00:00 2024-04-24T17:59:10.426 INFO:teuthology.orchestra.run.smithi078.stdout:(24/32): python3-zc-lockfile-2.0-2.el8.noarch.r 110 kB/s | 23 kB 00:00 2024-04-24T17:59:10.543 INFO:teuthology.orchestra.run.smithi078.stdout:(25/32): python3-markupsafe-0.23-19.el8.x86_64. 258 kB/s | 39 kB 00:00 2024-04-24T17:59:10.560 INFO:teuthology.orchestra.run.smithi078.stdout:(26/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 774 kB/s | 103 kB 00:00 2024-04-24T17:59:10.660 INFO:teuthology.orchestra.run.smithi078.stdout:(27/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.3 MB/s | 157 kB 00:00 2024-04-24T17:59:10.693 INFO:teuthology.orchestra.run.smithi078.stdout:(28/32): python3-pytz-2017.2-11.el8.noarch.rpm 406 kB/s | 54 kB 00:00 2024-04-24T17:59:10.835 INFO:teuthology.orchestra.run.smithi078.stdout:(29/32): python3-pycparser-2.14-14.el8.noarch.r 622 kB/s | 109 kB 00:00 2024-04-24T17:59:10.877 INFO:teuthology.orchestra.run.smithi078.stdout:(30/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-04-24T17:59:10.961 INFO:teuthology.orchestra.run.smithi078.stdout:(31/32): python3-cryptography-3.2.1-7.el8_9.x86 4.4 MB/s | 559 kB 00:00 2024-04-24T17:59:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 25/25 2024-04-24T17:59:12.484 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout:Upgraded: 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: librados2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: librbd1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-base-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-radosgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-selinux-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2024-04-24T17:59:13.059 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: liboath-2.6.2-3.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: librabbitmq-0.9.0-5.el8_9.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: libradosstriper1-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: librdkafka-0.11.4-3.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: librgw2-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: libunwind-1.3.1-3.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: lttng-ust-2.8.1-11.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-argparse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: python3-ceph-common-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: python3-prettytable-0.7.2-14.el8.noarch 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rados-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.060 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:13.061 INFO:teuthology.orchestra.run.smithi049.stdout: thrift-0.13.0-2.el8.x86_64 2024-04-24T17:59:13.061 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:13.061 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T17:59:13.409 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-test 2024-04-24T17:59:13.783 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T17:59:15.557 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 25 kB/s | 4.5 kB 00:00 2024-04-24T17:59:16.237 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-24T17:59:18.718 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================== 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================== 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test x86_64 2:17.2.7-915.g330067db.el8 ceph 53 M 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: jq x86_64 1.6-7.el8 rhel-8-for-x86_64-appstream-rpms 202 k 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite x86_64 2:17.2.7-915.g330067db.el8 ceph 179 k 2024-04-24T17:59:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma x86_64 6.8.2-2.1.el8_9 rhel-8-for-x86_64-appstream-rpms 188 k 2024-04-24T17:59:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: socat x86_64 1.7.4.1-1.el8 rhel-8-for-x86_64-appstream-rpms 323 k 2024-04-24T17:59:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet x86_64 1.6.1-20.el8 lab-extras 70 k 2024-04-24T17:59:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:18.720 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T17:59:18.720 INFO:teuthology.orchestra.run.smithi049.stdout:=========================================================================================== 2024-04-24T17:59:18.720 INFO:teuthology.orchestra.run.smithi049.stdout:Install 6 Packages 2024-04-24T17:59:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:18.721 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 54 M 2024-04-24T17:59:18.721 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 236 M 2024-04-24T17:59:18.721 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T17:59:18.751 INFO:teuthology.orchestra.run.smithi049.stdout:(1/6): xmlstarlet-1.6.1-20.el8.x86_64.rpm 2.3 MB/s | 70 kB 00:00 2024-04-24T17:59:18.952 INFO:teuthology.orchestra.run.smithi049.stdout:(2/6): socat-1.7.4.1-1.el8.x86_64.rpm 1.6 MB/s | 323 kB 00:00 2024-04-24T17:59:19.052 INFO:teuthology.orchestra.run.smithi049.stdout:(3/6): jq-1.6-7.el8.x86_64.rpm 2.0 MB/s | 202 kB 00:00 2024-04-24T17:59:19.177 INFO:teuthology.orchestra.run.smithi049.stdout:(4/6): oniguruma-6.8.2-2.1.el8_9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-24T17:59:19.669 INFO:teuthology.orchestra.run.smithi049.stdout:(5/6): libcephsqlite-17.2.7-915.g330067db.el8.x 189 kB/s | 179 kB 00:00 2024-04-24T17:59:25.202 INFO:teuthology.orchestra.run.smithi049.stdout:(6/6): ceph-test-17.2.7-915.g330067db.el8.x86_6 8.1 MB/s | 53 MB 00:06 2024-04-24T17:59:25.204 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T17:59:25.204 INFO:teuthology.orchestra.run.smithi049.stdout:Total 8.3 MB/s | 54 MB 00:06 2024-04-24T17:59:25.246 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T17:59:25.269 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T17:59:25.269 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T17:59:25.553 INFO:teuthology.orchestra.run.smithi078.stdout:(32/32): ceph-osd-17.2.7-915.g330067db.el8.x86_ 1.0 MB/s | 17 MB 00:17 2024-04-24T17:59:25.557 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T17:59:25.557 INFO:teuthology.orchestra.run.smithi078.stdout:Total 1.4 MB/s | 29 MB 00:19 2024-04-24T17:59:25.575 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T17:59:25.585 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T17:59:25.650 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T17:59:25.687 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T17:59:25.687 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T17:59:25.904 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T17:59:25.913 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T17:59:26.119 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T17:59:26.357 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-24T17:59:26.464 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: oniguruma-6.8.2-2.1.el8_9.x86_64 1/6 2024-04-24T17:59:26.655 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : jq-1.6-7.el8.x86_64 2/6 2024-04-24T17:59:26.858 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 3/6 2024-04-24T17:59:27.007 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 4/6 2024-04-24T17:59:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T17:59:27.182 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T17:59:27.234 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-24T17:59:27.457 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-24T17:59:27.681 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 3/32 2024-04-24T17:59:28.060 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 4/32 2024-04-24T17:59:28.298 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 5/32 2024-04-24T17:59:28.511 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 6/32 2024-04-24T17:59:28.679 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 7/32 2024-04-24T17:59:28.901 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 8/32 2024-04-24T17:59:29.074 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 9/32 2024-04-24T17:59:29.323 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 10/32 2024-04-24T17:59:29.659 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 11/32 2024-04-24T17:59:29.886 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 12/32 2024-04-24T17:59:30.095 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-24T17:59:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-24T17:59:30.457 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-24T17:59:30.688 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-24T17:59:30.856 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-24T17:59:31.005 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-24T17:59:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-24T17:59:31.423 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-24T17:59:31.609 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-24T17:59:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-24T17:59:32.116 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-24T17:59:32.337 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-24T17:59:32.521 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 5/6 2024-04-24T17:59:32.619 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-24T17:59:32.780 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T17:59:33.023 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-24T17:59:33.051 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 6/6 2024-04-24T17:59:33.051 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-test-2:17.2.7-915.g330067db.el8.x86_64 1/6 2024-04-24T17:59:33.051 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2/6 2024-04-24T17:59:33.051 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 3/6 2024-04-24T17:59:33.051 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 4/6 2024-04-24T17:59:33.051 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : jq-1.6-7.el8.x86_64 5/6 2024-04-24T17:59:33.406 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 27/32 2024-04-24T17:59:33.608 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T17:59:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T17:59:35.333 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T17:59:35.734 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T17:59:35.903 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T17:59:36.121 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T17:59:36.312 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T17:59:36.330 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T17:59:36.443 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T17:59:36.817 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T17:59:36.817 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-2:17.2.7-915.g330067db.el8.x86_64 1/32 2024-04-24T17:59:36.817 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2/32 2024-04-24T17:59:36.817 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 3/32 2024-04-24T17:59:36.817 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 4/32 2024-04-24T17:59:36.818 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 5/32 2024-04-24T17:59:36.818 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 6/32 2024-04-24T17:59:36.818 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-24T17:59:36.818 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-24T17:59:36.818 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-24T17:59:36.818 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-24T17:59:36.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-24T17:59:36.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-24T17:59:36.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-24T17:59:36.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-24T17:59:36.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-24T17:59:36.819 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-24T17:59:36.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-24T17:59:36.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-24T17:59:36.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-24T17:59:36.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-24T17:59:36.820 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-24T17:59:36.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-24T17:59:36.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-24T17:59:36.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-24T17:59:36.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/32 2024-04-24T17:59:36.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/32 2024-04-24T17:59:36.821 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 27/32 2024-04-24T17:59:36.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 28/32 2024-04-24T17:59:36.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 29/32 2024-04-24T17:59:36.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 30/32 2024-04-24T17:59:36.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 31/32 2024-04-24T17:59:43.028 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : oniguruma-6.8.2-2.1.el8_9.x86_64 6/6 2024-04-24T17:59:43.029 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T17:59:43.606 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-test-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: jq-1.6-7.el8.x86_64 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: libcephsqlite-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: oniguruma-6.8.2-2.1.el8_9.x86_64 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: socat-1.7.4.1-1.el8.x86_64 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:43.607 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T17:59:43.789 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph 2024-04-24T17:59:44.148 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T17:59:45.741 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 32/32 2024-04-24T17:59:45.741 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T17:59:45.909 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 34 kB/s | 4.5 kB 00:00 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-24T17:59:46.287 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-portend-2.6-1.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-24T17:59:46.288 INFO:teuthology.orchestra.run.smithi078.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:46.289 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T17:59:46.474 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-base 2024-04-24T17:59:46.573 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 kB/s | 4.1 kB 00:00 2024-04-24T17:59:46.824 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:59:48.575 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-24T17:59:49.079 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout:====================================================================================================== 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout:====================================================================================================== 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout: ceph x86_64 2:17.2.7-915.g330067db.el8 ceph 6.4 k 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds x86_64 2:17.2.7-915.g330067db.el8 ceph 2.3 M 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr x86_64 2:17.2.7-915.g330067db.el8 ceph 1.6 M 2024-04-24T17:59:49.083 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 240 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon x86_64 2:17.2.7-915.g330067db.el8 ceph 4.0 M 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd x86_64 2:17.2.7-915.g330067db.el8 ceph 17 M 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi x86_64 1.11.5-6.el8 rhel-8-for-x86_64-baseos-rpms 238 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography x86_64 3.2.1-7.el8_9 rhel-8-for-x86_64-baseos-rpms 559 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako noarch 1.0.6-14.el8 rhel-8-for-x86_64-appstream-rpms 157 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe x86_64 0.23-19.el8 rhel-8-for-x86_64-appstream-rpms 39 k 2024-04-24T17:59:49.084 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 rhel-8-for-x86_64-appstream-rpms 103 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser noarch 2.14-14.el8 rhel-8-for-x86_64-baseos-rpms 109 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz noarch 2017.2-11.el8 rhel-8-for-x86_64-appstream-rpms 54 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug noarch 0.12.2-4.el8 rhel-8-for-x86_64-appstream-rpms 457 k 2024-04-24T17:59:49.085 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2024-04-24T17:59:49.086 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:49.086 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T17:59:49.086 INFO:teuthology.orchestra.run.smithi049.stdout:====================================================================================================== 2024-04-24T17:59:49.086 INFO:teuthology.orchestra.run.smithi049.stdout:Install 32 Packages 2024-04-24T17:59:49.086 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T17:59:49.086 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 29 M 2024-04-24T17:59:49.086 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 113 M 2024-04-24T17:59:49.087 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T17:59:49.257 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-04-24T17:59:51.020 INFO:teuthology.orchestra.run.smithi049.stdout:(1/32): ceph-17.2.7-915.g330067db.el8.x86_64.rp 3.9 kB/s | 6.4 kB 00:01 2024-04-24T17:59:51.467 INFO:teuthology.orchestra.run.smithi078.stdout:Package ceph-base-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T17:59:51.653 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T17:59:51.654 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T17:59:51.655 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T17:59:51.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install cephadm 2024-04-24T17:59:52.139 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T17:59:52.778 INFO:teuthology.orchestra.run.smithi049.stdout:(2/32): ceph-mds-17.2.7-915.g330067db.el8.x86_6 700 kB/s | 2.3 MB 00:03 2024-04-24T17:59:54.093 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-24T17:59:54.782 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-24T17:59:55.928 INFO:teuthology.orchestra.run.smithi049.stdout:(3/32): ceph-mgr-17.2.7-915.g330067db.el8.x86_6 246 kB/s | 1.6 MB 00:06 2024-04-24T17:59:56.361 INFO:teuthology.orchestra.run.smithi049.stdout:(4/32): ceph-mgr-modules-core-17.2.7-915.g33006 554 kB/s | 240 kB 00:00 2024-04-24T17:59:56.545 INFO:teuthology.orchestra.run.smithi049.stdout:(5/32): ceph-mon-17.2.7-915.g330067db.el8.x86_6 746 kB/s | 4.0 MB 00:05 2024-04-24T17:59:56.712 INFO:teuthology.orchestra.run.smithi049.stdout:(6/32): python3-bcrypt-3.1.6-2.el8.1.x86_64.rpm 124 kB/s | 44 kB 00:00 2024-04-24T17:59:56.896 INFO:teuthology.orchestra.run.smithi049.stdout:(7/32): python3-cheroot-8.5.2-1.el8.noarch.rpm 941 kB/s | 173 kB 00:00 2024-04-24T17:59:56.921 INFO:teuthology.orchestra.run.smithi049.stdout:(8/32): python3-beautifulsoup4-4.6.3-2.el8.1.no 494 kB/s | 185 kB 00:00 2024-04-24T17:59:56.988 INFO:teuthology.orchestra.run.smithi049.stdout:(9/32): python3-jaraco-6.2-6.el8.noarch.rpm 167 kB/s | 11 kB 00:00 2024-04-24T17:59:57.046 INFO:teuthology.orchestra.run.smithi049.stdout:(10/32): python3-cherrypy-18.4.0-1.el8.noarch.r 2.5 MB/s | 384 kB 00:00 2024-04-24T17:59:57.063 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T17:59:57.063 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T17:59:57.063 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T17:59:57.063 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 82 k 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 82 k 2024-04-24T17:59:57.064 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 372 k 2024-04-24T17:59:57.065 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T17:59:57.072 INFO:teuthology.orchestra.run.smithi049.stdout:(11/32): python3-jaraco-functools-2.0-4.el8.noa 210 kB/s | 18 kB 00:00 2024-04-24T17:59:57.147 INFO:teuthology.orchestra.run.smithi049.stdout:(12/32): python3-logutils-0.3.5-11.el8.noarch.r 486 kB/s | 49 kB 00:00 2024-04-24T17:59:57.172 INFO:teuthology.orchestra.run.smithi049.stdout:(13/32): python3-more-itertools-7.2.0-3.el8.noa 587 kB/s | 59 kB 00:00 2024-04-24T17:59:57.239 INFO:teuthology.orchestra.run.smithi049.stdout:(14/32): python3-pecan-1.3.2-9.el8.noarch.rpm 3.0 MB/s | 283 kB 00:00 2024-04-24T17:59:57.256 INFO:teuthology.orchestra.run.smithi049.stdout:(15/32): python3-portend-2.6-1.el8.noarch.rpm 186 kB/s | 16 kB 00:00 2024-04-24T17:59:57.306 INFO:teuthology.orchestra.run.smithi049.stdout:(16/32): python3-simplegeneric-0.8.1-17.el8.noa 280 kB/s | 19 kB 00:00 2024-04-24T17:59:57.323 INFO:teuthology.orchestra.run.smithi049.stdout:(17/32): python3-singledispatch-3.4.0.3-18.el8. 358 kB/s | 24 kB 00:00 2024-04-24T17:59:57.373 INFO:teuthology.orchestra.run.smithi049.stdout:(18/32): python3-tempora-1.14.1-5.el8.noarch.rp 436 kB/s | 29 kB 00:00 2024-04-24T17:59:57.390 INFO:teuthology.orchestra.run.smithi049.stdout:(19/32): python3-trustme-0.6.0-4.el8.noarch.rpm 411 kB/s | 27 kB 00:00 2024-04-24T17:59:57.482 INFO:teuthology.orchestra.run.smithi049.stdout:(20/32): python3-waitress-1.4.3-1.el8.noarch.rp 2.3 MB/s | 255 kB 00:00 2024-04-24T17:59:57.508 INFO:teuthology.orchestra.run.smithi049.stdout:(21/32): python3-webob-1.8.5-1.el8.1.noarch.rpm 2.1 MB/s | 251 kB 00:00 2024-04-24T17:59:57.558 INFO:teuthology.orchestra.run.smithi049.stdout:(22/32): python3-webtest-2.0.33-1.el8.noarch.rp 1.1 MB/s | 85 kB 00:00 2024-04-24T17:59:57.592 INFO:teuthology.orchestra.run.smithi049.stdout:(23/32): python3-zc-lockfile-2.0-2.el8.noarch.r 274 kB/s | 23 kB 00:00 2024-04-24T17:59:57.610 INFO:teuthology.orchestra.run.smithi078.stdout:cephadm-17.2.7-915.g330067db.el8.noarch.rpm 150 kB/s | 82 kB 00:00 2024-04-24T17:59:57.611 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T17:59:57.611 INFO:teuthology.orchestra.run.smithi078.stdout:Total 150 kB/s | 82 kB 00:00 2024-04-24T17:59:57.612 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T17:59:57.620 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T17:59:57.620 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T17:59:57.622 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T17:59:57.631 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T17:59:57.672 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T17:59:57.802 INFO:teuthology.orchestra.run.smithi049.stdout:(24/32): python3-markupsafe-0.23-19.el8.x86_64. 185 kB/s | 39 kB 00:00 2024-04-24T17:59:57.867 INFO:teuthology.orchestra.run.smithi049.stdout:(25/32): python3-werkzeug-0.12.2-4.el8.noarch.r 1.4 MB/s | 457 kB 00:00 2024-04-24T17:59:57.917 INFO:teuthology.orchestra.run.smithi049.stdout:(26/32): python3-pyOpenSSL-19.0.0-1.el8.noarch. 901 kB/s | 103 kB 00:00 2024-04-24T17:59:57.993 INFO:teuthology.orchestra.run.smithi049.stdout:(27/32): python3-mako-1.0.6-14.el8.noarch.rpm 1.2 MB/s | 157 kB 00:00 2024-04-24T17:59:58.125 INFO:teuthology.orchestra.run.smithi049.stdout:(28/32): python3-pycparser-2.14-14.el8.noarch.r 830 kB/s | 109 kB 00:00 2024-04-24T17:59:58.152 INFO:teuthology.orchestra.run.smithi049.stdout:(29/32): python3-pytz-2017.2-11.el8.noarch.rpm 231 kB/s | 54 kB 00:00 2024-04-24T17:59:58.302 INFO:teuthology.orchestra.run.smithi049.stdout:(30/32): python3-cffi-1.11.5-6.el8.x86_64.rpm 1.3 MB/s | 238 kB 00:00 2024-04-24T17:59:58.336 INFO:teuthology.orchestra.run.smithi049.stdout:(31/32): python3-cryptography-3.2.1-7.el8_9.x86 3.0 MB/s | 559 kB 00:00 2024-04-24T17:59:59.817 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T17:59:59.967 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:00:00.167 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:00:07.955 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:00:07.955 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:00:08.492 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:08.492 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:00:08.492 INFO:teuthology.orchestra.run.smithi078.stdout: cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:00:08.492 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:08.492 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:00:08.690 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T18:00:09.054 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:00:10.778 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 45 kB/s | 4.5 kB 00:00 2024-04-24T18:00:11.522 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 22 kB/s | 4.1 kB 00:00 2024-04-24T18:00:12.151 INFO:teuthology.orchestra.run.smithi049.stdout:(32/32): ceph-osd-17.2.7-915.g330067db.el8.x86_ 906 kB/s | 17 MB 00:19 2024-04-24T18:00:12.155 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:00:12.156 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.3 MB/s | 29 MB 00:23 2024-04-24T18:00:12.253 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:00:12.293 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:00:12.293 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:00:12.528 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:00:12.537 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:00:13.707 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:00:13.816 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-915.g330067db.el8 ceph 161 k 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 161 k 2024-04-24T18:00:13.817 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 494 k 2024-04-24T18:00:13.818 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:00:13.926 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 1/32 2024-04-24T18:00:14.197 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 2/32 2024-04-24T18:00:14.438 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 3/32 2024-04-24T18:00:14.461 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-immutable-object-cache-17.2.7-915.g330067d 251 kB/s | 161 kB 00:00 2024-04-24T18:00:14.461 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:00:14.461 INFO:teuthology.orchestra.run.smithi078.stdout:Total 251 kB/s | 161 kB 00:00 2024-04-24T18:00:14.462 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:00:14.473 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:00:14.473 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:00:14.531 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:00:14.540 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:00:14.658 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:00:14.806 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T18:00:14.845 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 4/32 2024-04-24T18:00:15.128 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cryptography-3.2.1-7.el8_9.x86_64 5/32 2024-04-24T18:00:15.230 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T18:00:15.290 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 6/32 2024-04-24T18:00:15.467 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 7/32 2024-04-24T18:00:15.749 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 8/32 2024-04-24T18:00:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 9/32 2024-04-24T18:00:16.170 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 10/32 2024-04-24T18:00:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 11/32 2024-04-24T18:00:16.783 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 12/32 2024-04-24T18:00:17.028 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 13/32 2024-04-24T18:00:17.264 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 14/32 2024-04-24T18:00:17.470 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 15/32 2024-04-24T18:00:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 16/32 2024-04-24T18:00:17.844 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 17/32 2024-04-24T18:00:18.030 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 18/32 2024-04-24T18:00:18.241 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 19/32 2024-04-24T18:00:18.470 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 20/32 2024-04-24T18:00:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 21/32 2024-04-24T18:00:18.977 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-portend-2.6-1.el8.noarch 22/32 2024-04-24T18:00:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 23/32 2024-04-24T18:00:19.536 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 24/32 2024-04-24T18:00:19.927 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 25/32 2024-04-24T18:00:20.296 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 26/32 2024-04-24T18:00:20.709 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 27/32 2024-04-24T18:00:20.894 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T18:00:22.514 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 28/32 2024-04-24T18:00:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T18:00:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 29/32 2024-04-24T18:00:23.389 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T18:00:23.619 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 30/32 2024-04-24T18:00:23.980 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T18:00:24.000 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 31/32 2024-04-24T18:00:24.139 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-2:17.2.7-915.g330067db.el8.x86_64 32/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-2:17.2.7-915.g330067db.el8.x86_64 1/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 3/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 4/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 5/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.n 6/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 7/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 8/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 9/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 10/32 2024-04-24T18:00:24.556 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 11/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 12/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 13/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 14/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 15/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-portend-2.6-1.el8.noarch 16/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 17/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 18/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 19/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 20/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 21/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 22/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 23/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 24/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 25/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 26/32 2024-04-24T18:00:24.557 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 27/32 2024-04-24T18:00:24.558 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 28/32 2024-04-24T18:00:24.558 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 29/32 2024-04-24T18:00:24.558 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 30/32 2024-04-24T18:00:24.558 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 31/32 2024-04-24T18:00:25.064 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T18:00:25.064 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:00:25.620 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:25.621 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:00:25.621 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-immutable-object-cache-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:00:25.621 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:25.621 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:00:25.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr 2024-04-24T18:00:26.140 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:00:27.796 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 kB/s | 4.5 kB 00:00 2024-04-24T18:00:28.463 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-24T18:00:30.669 INFO:teuthology.orchestra.run.smithi078.stdout:Package ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:00:30.860 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:00:30.861 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:00:30.861 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:00:31.147 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T18:00:31.495 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:00:33.505 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 22 kB/s | 4.5 kB 00:00 2024-04-24T18:00:34.190 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-24T18:00:36.467 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================================== 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================================== 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 3.4 M 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 24 k 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 14 k 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:36.468 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:00:36.469 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================================== 2024-04-24T18:00:36.469 INFO:teuthology.orchestra.run.smithi078.stdout:Install 6 Packages 2024-04-24T18:00:36.469 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:36.469 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 3.7 M 2024-04-24T18:00:36.469 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 80 M 2024-04-24T18:00:36.469 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:00:37.931 INFO:teuthology.orchestra.run.smithi078.stdout:(1/6): ceph-grafana-dashboards-17.2.7-915.g3300 45 kB/s | 24 kB 00:00 2024-04-24T18:00:37.990 INFO:teuthology.orchestra.run.smithi078.stdout:(2/6): ceph-prometheus-alerts-17.2.7-915.g33006 24 kB/s | 14 kB 00:00 2024-04-24T18:00:38.548 INFO:teuthology.orchestra.run.smithi078.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 54 kB/s | 34 kB 00:00 2024-04-24T18:00:38.774 INFO:teuthology.orchestra.run.smithi078.stdout:(4/6): python3-jwt-1.6.1-2.el8.noarch.rpm 189 kB/s | 43 kB 00:00 2024-04-24T18:00:38.835 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cryptography-3.2.1-7.el8_9.x86_64 32/32 2024-04-24T18:00:38.835 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mds-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-modules-core-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mon-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-osd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cffi-1.11.5-6.el8.x86_64 2024-04-24T18:00:39.460 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cheroot-8.5.2-1.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cryptography-3.2.1-7.el8_9.x86_64 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-6.2-6.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-logutils-0.3.5-11.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-mako-1.0.6-14.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-markupsafe-0.23-19.el8.x86_64 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pecan-1.3.2-9.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-portend-2.6-1.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pycparser-2.14-14.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pytz-2017.2-11.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2024-04-24T18:00:39.461 INFO:teuthology.orchestra.run.smithi049.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: python3-tempora-1.14.1-5.el8.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: python3-trustme-0.6.0-4.el8.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: python3-waitress-1.4.3-1.el8.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webob-1.8.5-1.el8.1.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: python3-webtest-2.0.33-1.el8.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:00:39.462 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:00:39.650 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-base 2024-04-24T18:00:40.024 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:00:40.949 INFO:teuthology.orchestra.run.smithi078.stdout:(5/6): python3-routes-2.4.1-12.el8.noarch.rpm 66 kB/s | 196 kB 00:02 2024-04-24T18:00:42.016 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-24T18:00:42.688 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 36 kB/s | 4.1 kB 00:00 2024-04-24T18:00:45.036 INFO:teuthology.orchestra.run.smithi049.stdout:Package ceph-base-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:00:45.239 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:00:45.241 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:00:45.241 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:00:45.387 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install cephadm 2024-04-24T18:00:45.756 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:00:46.415 INFO:teuthology.orchestra.run.smithi078.stdout:(6/6): ceph-mgr-dashboard-17.2.7-915.g330067db. 380 kB/s | 3.4 MB 00:09 2024-04-24T18:00:46.417 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:00:46.417 INFO:teuthology.orchestra.run.smithi078.stdout:Total 376 kB/s | 3.7 MB 00:09 2024-04-24T18:00:46.430 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:00:46.440 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:00:46.441 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:00:46.517 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:00:46.526 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:00:46.831 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:00:47.011 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2024-04-24T18:00:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2024-04-24T18:00:47.404 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2024-04-24T18:00:47.558 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 4/6 2024-04-24T18:00:47.678 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-24T18:00:48.391 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 30 kB/s | 4.1 kB 00:00 2024-04-24T18:00:48.635 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 5/6 2024-04-24T18:00:48.873 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T18:00:49.134 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T18:00:49.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 1/6 2024-04-24T18:00:49.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2/6 2024-04-24T18:00:49.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 3/6 2024-04-24T18:00:49.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2024-04-24T18:00:49.134 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2024-04-24T18:00:50.942 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:00:50.942 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:00:50.942 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 82 k 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:00:50.943 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:00:50.944 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 82 k 2024-04-24T18:00:50.944 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 372 k 2024-04-24T18:00:50.944 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:00:51.118 INFO:teuthology.orchestra.run.smithi049.stdout:cephadm-17.2.7-915.g330067db.el8.noarch.rpm 474 kB/s | 82 kB 00:00 2024-04-24T18:00:51.118 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:00:51.118 INFO:teuthology.orchestra.run.smithi049.stdout:Total 470 kB/s | 82 kB 00:00 2024-04-24T18:00:51.118 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:00:51.127 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:00:51.127 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:00:51.129 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:00:51.138 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:00:51.185 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:00:53.179 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:00:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:00:53.587 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:00:57.871 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2024-04-24T18:00:57.871 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:00:58.439 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:58.439 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:00:58.439 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:00:58.439 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:00:58.439 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:00:58.439 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-24T18:00:58.439 INFO:teuthology.orchestra.run.smithi078.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-24T18:00:58.440 INFO:teuthology.orchestra.run.smithi078.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-24T18:00:58.440 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:00:58.440 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:00:58.651 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T18:00:59.003 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:01:00.837 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 46 kB/s | 4.5 kB 00:00 2024-04-24T18:01:01.508 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-24T18:01:03.931 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================================= 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================================= 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 7.4 M 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-24T18:01:03.932 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================================= 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout:Install 8 Packages 2024-04-24T18:01:03.933 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:01:03.934 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 36 M 2024-04-24T18:01:03.934 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 207 M 2024-04-24T18:01:03.934 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:01:04.131 INFO:teuthology.orchestra.run.smithi078.stdout:(1/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.1 MB/s | 225 kB 00:00 2024-04-24T18:01:04.248 INFO:teuthology.orchestra.run.smithi078.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 12 MB/s | 3.7 MB 00:00 2024-04-24T18:01:04.390 INFO:teuthology.orchestra.run.smithi078.stdout:(3/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 20 MB/s | 5.2 MB 00:00 2024-04-24T18:01:04.515 INFO:teuthology.orchestra.run.smithi078.stdout:(4/8): openblas-0.3.15-6.el8.x86_64.rpm 19 MB/s | 5.0 MB 00:00 2024-04-24T18:01:04.657 INFO:teuthology.orchestra.run.smithi078.stdout:(5/8): libgfortran-8.5.0-20.el8.x86_64.rpm 4.4 MB/s | 645 kB 00:00 2024-04-24T18:01:04.807 INFO:teuthology.orchestra.run.smithi078.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.9.0+20 33 MB/s | 14 MB 00:00 2024-04-24T18:01:04.832 INFO:teuthology.orchestra.run.smithi078.stdout:(7/8): libquadmath-8.5.0-20.el8.x86_64.rpm 985 kB/s | 172 kB 00:00 2024-04-24T18:01:05.776 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : cephadm-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:01:05.776 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:01:06.361 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:06.361 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:01:06.361 INFO:teuthology.orchestra.run.smithi049.stdout: cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:01:06.361 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:06.361 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:01:06.520 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-immutable-object-cache 2024-04-24T18:01:06.904 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:01:09.013 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 50 kB/s | 4.5 kB 00:00 2024-04-24T18:01:09.709 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-24T18:01:12.248 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-915.g330067db.el8 ceph 161 k 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:01:12.249 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:12.250 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 161 k 2024-04-24T18:01:12.250 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 494 k 2024-04-24T18:01:12.250 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:01:12.739 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-immutable-object-cache-17.2.7-915.g330067d 330 kB/s | 161 kB 00:00 2024-04-24T18:01:12.740 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:01:12.740 INFO:teuthology.orchestra.run.smithi049.stdout:Total 329 kB/s | 161 kB 00:00 2024-04-24T18:01:12.740 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:01:12.752 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:01:12.752 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:01:12.811 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:01:12.820 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:01:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:01:13.089 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T18:01:13.340 INFO:teuthology.orchestra.run.smithi078.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.7-915 807 kB/s | 7.4 MB 00:09 2024-04-24T18:01:13.341 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:01:13.341 INFO:teuthology.orchestra.run.smithi078.stdout:Total 3.8 MB/s | 36 MB 00:09 2024-04-24T18:01:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T18:01:13.591 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:01:13.609 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:01:13.609 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:01:13.919 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:01:13.928 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:01:14.512 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:01:14.647 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-24T18:01:14.795 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-24T18:01:14.940 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-24T18:01:15.566 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-24T18:01:15.756 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-24T18:01:16.362 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-24T18:01:16.598 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-24T18:01:17.284 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-24T18:01:17.560 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-24T18:01:17.739 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T18:01:19.506 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T18:01:20.815 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-24T18:01:21.005 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 1/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 2/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 3/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 4/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 5/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 6/8 2024-04-24T18:01:21.329 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2024-04-24T18:01:26.581 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-915.g330067db.e 1/1 2024-04-24T18:01:26.582 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:01:27.164 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:27.164 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:01:27.164 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-immutable-object-cache-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:01:27.164 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:27.165 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:01:27.364 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr 2024-04-24T18:01:27.734 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:01:29.613 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-24T18:01:30.307 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 38 kB/s | 4.1 kB 00:00 2024-04-24T18:01:32.653 INFO:teuthology.orchestra.run.smithi049.stdout:Package ceph-mgr-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:01:32.854 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:01:32.855 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:01:32.855 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:01:33.001 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-dashboard 2024-04-24T18:01:33.370 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:01:33.822 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2024-04-24T18:01:33.822 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:01:34.379 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:01:34.379 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:01:34.379 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:01:34.379 INFO:teuthology.orchestra.run.smithi078.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-24T18:01:34.379 INFO:teuthology.orchestra.run.smithi078.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-24T18:01:34.380 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-24T18:01:34.380 INFO:teuthology.orchestra.run.smithi078.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-24T18:01:34.380 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-24T18:01:34.380 INFO:teuthology.orchestra.run.smithi078.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-24T18:01:34.380 INFO:teuthology.orchestra.run.smithi078.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-24T18:01:34.380 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:01:34.380 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:01:34.597 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-rook 2024-04-24T18:01:34.948 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:01:35.029 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-04-24T18:01:35.709 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-24T18:01:36.686 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-24T18:01:37.311 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 90 kB/s | 4.1 kB 00:00 2024-04-24T18:01:38.260 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:01:38.261 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================================== 2024-04-24T18:01:38.261 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T18:01:38.261 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================================== 2024-04-24T18:01:38.261 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:01:38.261 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 3.4 M 2024-04-24T18:01:38.261 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 24 k 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 14 k 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt noarch 1.6.1-2.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================================== 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout:Install 6 Packages 2024-04-24T18:01:38.262 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:38.263 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 3.7 M 2024-04-24T18:01:38.263 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 80 M 2024-04-24T18:01:38.263 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:01:38.868 INFO:teuthology.orchestra.run.smithi049.stdout:(1/6): ceph-grafana-dashboards-17.2.7-915.g3300 164 kB/s | 24 kB 00:00 2024-04-24T18:01:38.893 INFO:teuthology.orchestra.run.smithi049.stdout:(2/6): ceph-prometheus-alerts-17.2.7-915.g33006 84 kB/s | 14 kB 00:00 2024-04-24T18:01:39.060 INFO:teuthology.orchestra.run.smithi049.stdout:(3/6): python3-repoze-lru-0.7-6.el8.noarch.rpm 175 kB/s | 34 kB 00:00 2024-04-24T18:01:39.160 INFO:teuthology.orchestra.run.smithi049.stdout:(4/6): python3-routes-2.4.1-12.el8.noarch.rpm 736 kB/s | 196 kB 00:00 2024-04-24T18:01:39.210 INFO:teuthology.orchestra.run.smithi049.stdout:(5/6): python3-jwt-1.6.1-2.el8.noarch.rpm 284 kB/s | 43 kB 00:00 2024-04-24T18:01:39.610 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================= 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================= 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 47 k 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-24T18:01:39.612 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout:======================================================================================================= 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout:Install 13 Packages 2024-04-24T18:01:39.613 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:01:39.614 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 1.9 M 2024-04-24T18:01:39.614 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 25 M 2024-04-24T18:01:39.614 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:01:41.002 INFO:teuthology.orchestra.run.smithi049.stdout:(6/6): ceph-mgr-dashboard-17.2.7-915.g330067db. 1.5 MB/s | 3.4 MB 00:02 2024-04-24T18:01:41.006 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:01:41.006 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.3 MB/s | 3.7 MB 00:02 2024-04-24T18:01:41.021 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:01:41.032 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:01:41.032 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:01:41.120 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:01:41.129 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:01:41.437 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:01:41.645 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 1/6 2024-04-24T18:01:41.843 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 2/6 2024-04-24T18:01:42.070 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 3/6 2024-04-24T18:01:42.221 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 4/6 2024-04-24T18:01:43.364 INFO:teuthology.orchestra.run.smithi078.stdout:(1/13): ceph-mgr-rook-17.2.7-915.g330067db.el8. 87 kB/s | 47 kB 00:00 2024-04-24T18:01:43.388 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 5/6 2024-04-24T18:01:43.397 INFO:teuthology.orchestra.run.smithi078.stdout:(2/13): python3-certifi-2018.10.15-7.el8.noarch 28 kB/s | 16 kB 00:00 2024-04-24T18:01:43.414 INFO:teuthology.orchestra.run.smithi078.stdout:(3/13): python3-cachetools-3.1.1-4.el8.noarch.r 55 kB/s | 33 kB 00:00 2024-04-24T18:01:43.638 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T18:01:43.639 INFO:teuthology.orchestra.run.smithi078.stdout:(4/13): python3-rsa-4.9-2.el8.noarch.rpm 285 kB/s | 64 kB 00:00 2024-04-24T18:01:43.856 INFO:teuthology.orchestra.run.smithi078.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 282 kB/s | 61 kB 00:00 2024-04-24T18:01:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 6/6 2024-04-24T18:01:43.922 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.n 1/6 2024-04-24T18:01:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2/6 2024-04-24T18:01:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.no 3/6 2024-04-24T18:01:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 4/6 2024-04-24T18:01:43.923 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 5/6 2024-04-24T18:01:43.982 INFO:teuthology.orchestra.run.smithi078.stdout:(6/13): python3-jsonpointer-1.10-11.el8.noarch. 160 kB/s | 20 kB 00:00 2024-04-24T18:01:44.057 INFO:teuthology.orchestra.run.smithi078.stdout:(7/13): python3-google-auth-1.1.1-10.el8.noarch 117 kB/s | 81 kB 00:00 2024-04-24T18:01:44.091 INFO:teuthology.orchestra.run.smithi078.stdout:(8/13): python3-pyasn1-modules-0.3.7-6.el8.noar 1.0 MB/s | 110 kB 00:00 2024-04-24T18:01:44.200 INFO:teuthology.orchestra.run.smithi078.stdout:(9/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.2 MB/s | 126 kB 00:00 2024-04-24T18:01:44.308 INFO:teuthology.orchestra.run.smithi078.stdout:(10/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 1.4 MB/s | 155 kB 00:00 2024-04-24T18:01:44.350 INFO:teuthology.orchestra.run.smithi078.stdout:(11/13): python3-jsonpatch-1.21-2.el8.noarch.rp 91 kB/s | 27 kB 00:00 2024-04-24T18:01:44.400 INFO:teuthology.orchestra.run.smithi078.stdout:(12/13): python3-requests-oauthlib-1.0.0-1.el8. 465 kB/s | 43 kB 00:00 2024-04-24T18:01:45.109 INFO:teuthology.orchestra.run.smithi078.stdout:(13/13): python3-kubernetes-11.0.0-6.el8.noarch 698 kB/s | 1.2 MB 00:01 2024-04-24T18:01:45.111 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:01:45.112 INFO:teuthology.orchestra.run.smithi078.stdout:Total 360 kB/s | 1.9 MB 00:05 2024-04-24T18:01:45.177 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:01:45.192 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:01:45.192 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:01:45.273 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:01:45.282 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:01:45.760 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:01:45.994 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-24T18:01:46.179 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/13 2024-04-24T18:01:46.380 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2024-04-24T18:01:46.648 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2024-04-24T18:01:46.858 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2024-04-24T18:01:47.050 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2024-04-24T18:01:47.263 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2024-04-24T18:01:47.450 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2024-04-24T18:01:47.658 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-24T18:01:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2024-04-24T18:01:48.980 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2024-04-24T18:01:49.979 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2024-04-24T18:01:50.143 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 1/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 8/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 9/13 2024-04-24T18:01:50.431 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2024-04-24T18:01:50.432 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/13 2024-04-24T18:01:50.432 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2024-04-24T18:01:55.602 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 6/6 2024-04-24T18:01:55.602 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:01:56.213 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-grafana-dashboards-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-dashboard-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-prometheus-alerts-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jwt-1.6.1-2.el8.noarch 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout: python3-repoze-lru-0.7-6.el8.noarch 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout: python3-routes-2.4.1-12.el8.noarch 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:01:56.214 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:01:56.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-24T18:01:56.765 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:01:58.509 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 38 kB/s | 4.5 kB 00:00 2024-04-24T18:01:59.193 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-24T18:02:01.751 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:02:01.752 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================================================= 2024-04-24T18:02:01.752 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T18:02:01.752 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================================================= 2024-04-24T18:02:01.752 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:02:01.752 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 7.4 M 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 645 k 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath x86_64 8.5.0-20.el8 rhel-8-for-x86_64-baseos-rpms 172 k 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: openblas x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.0 M 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads x86_64 0.3.15-6.el8 rhel-8-for-x86_64-appstream-rpms 5.2 M 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 3.7 M 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 rhel-8-for-x86_64-appstream-rpms 225 k 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy x86_64 1.0.0-21.module+el8.9.0+20976+d3c38525 rhel-8-for-x86_64-appstream-rpms 14 M 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================================================= 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout:Install 8 Packages 2024-04-24T18:02:01.753 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:01.754 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 36 M 2024-04-24T18:02:01.754 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 207 M 2024-04-24T18:02:01.754 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:02:01.876 INFO:teuthology.orchestra.run.smithi049.stdout:(1/8): python3-numpy-f2py-1.14.3-10.el8.x86_64. 1.8 MB/s | 225 kB 00:00 2024-04-24T18:02:01.968 INFO:teuthology.orchestra.run.smithi049.stdout:(2/8): python3-numpy-1.14.3-10.el8.x86_64.rpm 18 MB/s | 3.7 MB 00:00 2024-04-24T18:02:02.130 INFO:teuthology.orchestra.run.smithi049.stdout:(3/8): openblas-threads-0.3.15-6.el8.x86_64.rpm 21 MB/s | 5.2 MB 00:00 2024-04-24T18:02:02.186 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2024-04-24T18:02:02.186 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:02:02.251 INFO:teuthology.orchestra.run.smithi049.stdout:(4/8): openblas-0.3.15-6.el8.x86_64.rpm 18 MB/s | 5.0 MB 00:00 2024-04-24T18:02:02.385 INFO:teuthology.orchestra.run.smithi049.stdout:(5/8): libgfortran-8.5.0-20.el8.x86_64.rpm 4.7 MB/s | 645 kB 00:00 2024-04-24T18:02:02.627 INFO:teuthology.orchestra.run.smithi049.stdout:(6/8): python3-scipy-1.0.0-21.module+el8.9.0+20 28 MB/s | 14 MB 00:00 2024-04-24T18:02:02.652 INFO:teuthology.orchestra.run.smithi049.stdout:(7/8): libquadmath-8.5.0-20.el8.x86_64.rpm 646 kB/s | 172 kB 00:00 2024-04-24T18:02:02.776 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:02.776 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:02:02.776 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:02:02.776 INFO:teuthology.orchestra.run.smithi078.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-24T18:02:02.776 INFO:teuthology.orchestra.run.smithi078.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-24T18:02:02.776 INFO:teuthology.orchestra.run.smithi078.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:02.777 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:02:02.989 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T18:02:03.346 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:02:04.502 INFO:teuthology.orchestra.run.smithi049.stdout:(8/8): ceph-mgr-diskprediction-local-17.2.7-915 2.7 MB/s | 7.4 MB 00:02 2024-04-24T18:02:04.503 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:02:04.504 INFO:teuthology.orchestra.run.smithi049.stdout:Total 13 MB/s | 36 MB 00:02 2024-04-24T18:02:04.765 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:02:04.785 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:02:04.785 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:02:05.114 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:02:05.123 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:02:05.335 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 36 kB/s | 4.5 kB 00:00 2024-04-24T18:02:05.691 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:02:05.855 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-24T18:02:05.994 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 1/8 2024-04-24T18:02:05.997 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-24T18:02:06.155 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-24T18:02:06.823 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 2/8 2024-04-24T18:02:06.987 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-24T18:02:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-threads-0.3.15-6.el8.x86_64 3/8 2024-04-24T18:02:07.822 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : openblas-0.3.15-6.el8.x86_64 4/8 2024-04-24T18:02:08.288 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:02:08.289 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================================== 2024-04-24T18:02:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T18:02:08.289 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================================== 2024-04-24T18:02:08.289 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:02:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 123 k 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout:============================================================================================== 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout:Install 5 Packages 2024-04-24T18:02:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:08.291 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 6.0 M 2024-04-24T18:02:08.291 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 26 M 2024-04-24T18:02:08.291 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:02:08.541 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: openblas-0.3.15-6.el8.x86_64 4/8 2024-04-24T18:02:08.863 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 5/8 2024-04-24T18:02:09.088 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T18:02:09.419 INFO:teuthology.orchestra.run.smithi078.stdout:(1/5): ceph-mgr-cephadm-17.2.7-915.g330067db.el 647 kB/s | 123 kB 00:00 2024-04-24T18:02:09.444 INFO:teuthology.orchestra.run.smithi078.stdout:(2/5): python3-natsort-7.1.1-2.el8.noarch.rpm 279 kB/s | 60 kB 00:00 2024-04-24T18:02:09.570 INFO:teuthology.orchestra.run.smithi078.stdout:(3/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.4 MB/s | 476 kB 00:00 2024-04-24T18:02:09.620 INFO:teuthology.orchestra.run.smithi078.stdout:(4/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 3.0 MB/s | 538 kB 00:00 2024-04-24T18:02:09.837 INFO:teuthology.orchestra.run.smithi078.stdout:(5/5): python3-babel-2.5.1-7.el8.noarch.rpm 11 MB/s | 4.8 MB 00:00 2024-04-24T18:02:09.838 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:02:09.838 INFO:teuthology.orchestra.run.smithi078.stdout:Total 3.8 MB/s | 6.0 MB 00:01 2024-04-24T18:02:09.894 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:02:09.906 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:02:09.906 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:02:09.996 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:02:10.004 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:02:10.965 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:02:10.970 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 6/8 2024-04-24T18:02:11.277 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-24T18:02:11.534 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-24T18:02:11.822 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-24T18:02:12.126 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-24T18:02:12.282 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T18:02:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 7/8 2024-04-24T18:02:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T18:02:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 1/5 2024-04-24T18:02:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-24T18:02:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-24T18:02:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-24T18:02:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T18:02:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 8/8 2024-04-24T18:02:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db 1/8 2024-04-24T18:02:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 2/8 2024-04-24T18:02:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 3/8 2024-04-24T18:02:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-threads-0.3.15-6.el8.x86_64 4/8 2024-04-24T18:02:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : openblas-0.3.15-6.el8.x86_64 5/8 2024-04-24T18:02:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525 6/8 2024-04-24T18:02:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 7/8 2024-04-24T18:02:24.366 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-24T18:02:24.366 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:24.937 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:02:25.104 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-fuse 2024-04-24T18:02:25.463 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:02:25.581 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 8/8 2024-04-24T18:02:25.581 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:02:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:26.175 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:02:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-diskprediction-local-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:02:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: libgfortran-8.5.0-20.el8.x86_64 2024-04-24T18:02:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: libquadmath-8.5.0-20.el8.x86_64 2024-04-24T18:02:26.175 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-0.3.15-6.el8.x86_64 2024-04-24T18:02:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: openblas-threads-0.3.15-6.el8.x86_64 2024-04-24T18:02:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2024-04-24T18:02:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2024-04-24T18:02:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: python3-scipy-1.0.0-21.module+el8.9.0+20976+d3c38525.x86_64 2024-04-24T18:02:26.176 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:26.176 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:02:26.367 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-rook 2024-04-24T18:02:26.739 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:02:27.223 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 16 kB/s | 4.5 kB 00:00 2024-04-24T18:02:27.921 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-24T18:02:28.529 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 48 kB/s | 4.5 kB 00:00 2024-04-24T18:02:29.451 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 31 kB/s | 4.1 kB 00:00 2024-04-24T18:02:30.244 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:02:30.244 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:02:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T18:02:30.244 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: x86_64 2:17.2.7-915.g330067db.el8 ceph 876 k 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout:Installing dependencies: 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout:Install 2 Packages 2024-04-24T18:02:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:30.246 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 959 k 2024-04-24T18:02:30.246 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 3.1 M 2024-04-24T18:02:30.246 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:02:30.372 INFO:teuthology.orchestra.run.smithi078.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 659 kB/s | 83 kB 00:00 2024-04-24T18:02:30.489 INFO:teuthology.orchestra.run.smithi078.stdout:(2/2): ceph-fuse-17.2.7-915.g330067db.el8.x86_6 3.5 MB/s | 876 kB 00:00 2024-04-24T18:02:30.489 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:02:30.490 INFO:teuthology.orchestra.run.smithi078.stdout:Total 3.8 MB/s | 959 kB 00:00 2024-04-24T18:02:30.501 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:02:30.522 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:02:30.523 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:02:30.609 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:02:30.617 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:02:30.789 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:02:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-24T18:02:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T18:02:31.575 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T18:02:31.576 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/2 2024-04-24T18:02:32.020 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:02:32.021 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================================= 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================================= 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 47 k 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch noarch 1.21-2.el8 rhel-8-for-x86_64-appstream-rpms 27 k 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer noarch 1.10-11.el8 rhel-8-for-x86_64-appstream-rpms 20 k 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib noarch 2.1.0-1.el8 rhel-8-for-x86_64-baseos-rpms 155 k 2024-04-24T18:02:32.022 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1 noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 126 k 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 rhel-8-for-x86_64-appstream-rpms 110 k 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 rhel-8-for-x86_64-baseos-rpms 43 k 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout:======================================================================================================= 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout:Install 13 Packages 2024-04-24T18:02:32.023 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:32.024 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 1.9 M 2024-04-24T18:02:32.024 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 25 M 2024-04-24T18:02:32.024 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:02:32.292 INFO:teuthology.orchestra.run.smithi049.stdout:(1/13): python3-certifi-2018.10.15-7.el8.noarch 92 kB/s | 16 kB 00:00 2024-04-24T18:02:32.318 INFO:teuthology.orchestra.run.smithi049.stdout:(2/13): python3-cachetools-3.1.1-4.el8.noarch.r 162 kB/s | 33 kB 00:00 2024-04-24T18:02:32.360 INFO:teuthology.orchestra.run.smithi049.stdout:(3/13): ceph-mgr-rook-17.2.7-915.g330067db.el8. 193 kB/s | 47 kB 00:00 2024-04-24T18:02:32.401 INFO:teuthology.orchestra.run.smithi049.stdout:(4/13): python3-google-auth-1.1.1-10.el8.noarch 745 kB/s | 81 kB 00:00 2024-04-24T18:02:32.469 INFO:teuthology.orchestra.run.smithi049.stdout:(5/13): python3-websocket-client-0.56.0-5.el8.n 916 kB/s | 61 kB 00:00 2024-04-24T18:02:32.594 INFO:teuthology.orchestra.run.smithi049.stdout:(6/13): python3-rsa-4.9-2.el8.noarch.rpm 274 kB/s | 64 kB 00:00 2024-04-24T18:02:32.636 INFO:teuthology.orchestra.run.smithi049.stdout:(7/13): python3-jsonpointer-1.10-11.el8.noarch. 120 kB/s | 20 kB 00:00 2024-04-24T18:02:32.669 INFO:teuthology.orchestra.run.smithi049.stdout:(8/13): python3-kubernetes-11.0.0-6.el8.noarch. 3.3 MB/s | 1.2 MB 00:00 2024-04-24T18:02:32.728 INFO:teuthology.orchestra.run.smithi049.stdout:(9/13): python3-jsonpatch-1.21-2.el8.noarch.rpm 289 kB/s | 27 kB 00:00 2024-04-24T18:02:32.861 INFO:teuthology.orchestra.run.smithi049.stdout:(10/13): python3-pyasn1-modules-0.3.7-6.el8.noa 411 kB/s | 110 kB 00:00 2024-04-24T18:02:32.920 INFO:teuthology.orchestra.run.smithi049.stdout:(11/13): python3-oauthlib-2.1.0-1.el8.noarch.rp 807 kB/s | 155 kB 00:00 2024-04-24T18:02:32.937 INFO:teuthology.orchestra.run.smithi049.stdout:(12/13): python3-pyasn1-0.3.7-6.el8.noarch.rpm 473 kB/s | 126 kB 00:00 2024-04-24T18:02:33.046 INFO:teuthology.orchestra.run.smithi049.stdout:(13/13): python3-requests-oauthlib-1.0.0-1.el8. 232 kB/s | 43 kB 00:00 2024-04-24T18:02:33.046 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:02:33.046 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.9 MB/s | 1.9 MB 00:01 2024-04-24T18:02:33.114 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:02:33.131 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:02:33.131 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:02:33.217 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:02:33.226 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:02:33.749 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:02:34.037 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 1/13 2024-04-24T18:02:34.325 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-rsa-4.9-2.el8.noarch 2/13 2024-04-24T18:02:34.554 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 3/13 2024-04-24T18:02:34.791 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 4/13 2024-04-24T18:02:35.033 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 5/13 2024-04-24T18:02:35.375 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 6/13 2024-04-24T18:02:35.557 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 7/13 2024-04-24T18:02:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 8/13 2024-04-24T18:02:36.042 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 9/13 2024-04-24T18:02:36.285 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 10/13 2024-04-24T18:02:37.432 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 11/13 2024-04-24T18:02:37.710 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 12/13 2024-04-24T18:02:37.858 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T18:02:38.177 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 13/13 2024-04-24T18:02:38.177 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 1/13 2024-04-24T18:02:38.177 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 2/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 3/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 4/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 5/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 6/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 7/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 8/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 9/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 10/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 11/13 2024-04-24T18:02:38.178 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 12/13 2024-04-24T18:02:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-24T18:02:42.268 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:02:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:42.833 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:02:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-24T18:02:42.833 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:42.833 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:02:43.012 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install ceph-volume 2024-04-24T18:02:43.372 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:02:45.482 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-04-24T18:02:46.179 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-04-24T18:02:48.478 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:02:48.478 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repository Size 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 267 k 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 267 k 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 1.3 M 2024-04-24T18:02:48.479 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:02:48.803 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-volume-17.2.7-915.g330067db.el8.noarch.rpm 827 kB/s | 267 kB 00:00 2024-04-24T18:02:48.803 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:02:48.804 INFO:teuthology.orchestra.run.smithi078.stdout:Total 824 kB/s | 267 kB 00:00 2024-04-24T18:02:48.804 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:02:48.814 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:02:48.814 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:02:48.842 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:02:48.852 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:02:49.036 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:02:49.269 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:02:49.666 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:02:51.591 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 13/13 2024-04-24T18:02:51.591 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:02:52.192 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:52.192 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:02:52.192 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-rook-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:02:52.192 INFO:teuthology.orchestra.run.smithi049.stdout: python3-cachetools-3.1.1-4.el8.noarch 2024-04-24T18:02:52.192 INFO:teuthology.orchestra.run.smithi049.stdout: python3-certifi-2018.10.15-7.el8.noarch 2024-04-24T18:02:52.192 INFO:teuthology.orchestra.run.smithi049.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpatch-1.21-2.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jsonpointer-1.10-11.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-rsa-4.9-2.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:52.193 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:02:52.416 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-mgr-cephadm 2024-04-24T18:02:52.794 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:02:54.651 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 27 kB/s | 4.5 kB 00:00 2024-04-24T18:02:55.287 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 58 kB/s | 4.1 kB 00:00 2024-04-24T18:02:57.885 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================== 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================== 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 123 k 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel noarch 2.5.1-7.el8 rhel-8-for-x86_64-appstream-rpms 4.8 M 2024-04-24T18:02:57.886 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2 noarch 2.10.1-3.el8 rhel-8-for-x86_64-appstream-rpms 538 k 2024-04-24T18:02:57.887 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2024-04-24T18:02:57.887 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:57.887 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:02:57.887 INFO:teuthology.orchestra.run.smithi049.stdout:============================================================================================== 2024-04-24T18:02:57.887 INFO:teuthology.orchestra.run.smithi049.stdout:Install 5 Packages 2024-04-24T18:02:57.887 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:02:57.887 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 6.0 M 2024-04-24T18:02:57.888 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 26 M 2024-04-24T18:02:57.888 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:02:58.133 INFO:teuthology.orchestra.run.smithi049.stdout:(1/5): python3-natsort-7.1.1-2.el8.noarch.rpm 321 kB/s | 60 kB 00:00 2024-04-24T18:02:58.200 INFO:teuthology.orchestra.run.smithi049.stdout:(2/5): python3-asyncssh-2.7.0-2.el8.noarch.rpm 1.8 MB/s | 476 kB 00:00 2024-04-24T18:02:58.384 INFO:teuthology.orchestra.run.smithi049.stdout:(3/5): python3-babel-2.5.1-7.el8.noarch.rpm 19 MB/s | 4.8 MB 00:00 2024-04-24T18:02:58.442 INFO:teuthology.orchestra.run.smithi049.stdout:(4/5): ceph-mgr-cephadm-17.2.7-915.g330067db.el 248 kB/s | 123 kB 00:00 2024-04-24T18:02:58.476 INFO:teuthology.orchestra.run.smithi049.stdout:(5/5): python3-jinja2-2.10.1-3.el8.noarch.rpm 1.9 MB/s | 538 kB 00:00 2024-04-24T18:02:58.478 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:02:58.478 INFO:teuthology.orchestra.run.smithi049.stdout:Total 10 MB/s | 6.0 MB 00:00 2024-04-24T18:02:58.533 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:02:58.545 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:02:58.545 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:02:58.639 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:02:58.648 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:02:59.709 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:03:00.051 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 1/5 2024-04-24T18:03:00.450 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 2/5 2024-04-24T18:03:00.781 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-24T18:03:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 4/5 2024-04-24T18:03:01.213 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T18:03:01.544 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 5/5 2024-04-24T18:03:01.544 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 1/5 2024-04-24T18:03:01.544 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 2/5 2024-04-24T18:03:01.544 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 3/5 2024-04-24T18:03:01.544 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 4/5 2024-04-24T18:03:05.219 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:03:05.219 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:03:05.773 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:05.773 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:03:05.773 INFO:teuthology.orchestra.run.smithi078.stdout: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:03:05.773 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:05.773 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:03:05.891 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install librados-devel 2024-04-24T18:03:06.250 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:08.146 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 56 kB/s | 4.5 kB 00:00 2024-04-24T18:03:08.828 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 32 kB/s | 4.1 kB 00:00 2024-04-24T18:03:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 5/5 2024-04-24T18:03:10.799 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:03:11.133 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:03:11.133 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:03:11.133 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-24T18:03:11.133 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:03:11.134 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:03:11.134 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 130 k 2024-04-24T18:03:11.134 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:11.134 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:03:11.134 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:03:11.134 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T18:03:11.134 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:11.135 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 130 k 2024-04-24T18:03:11.135 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 464 k 2024-04-24T18:03:11.135 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:03:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:11.408 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:03:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-mgr-cephadm-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:03:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2024-04-24T18:03:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: python3-babel-2.5.1-7.el8.noarch 2024-04-24T18:03:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: python3-jinja2-2.10.1-3.el8.noarch 2024-04-24T18:03:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: python3-natsort-7.1.1-2.el8.noarch 2024-04-24T18:03:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:11.409 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:03:11.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-fuse 2024-04-24T18:03:11.749 INFO:teuthology.orchestra.run.smithi078.stdout:librados-devel-17.2.7-915.g330067db.el8.x86_64. 211 kB/s | 130 kB 00:00 2024-04-24T18:03:11.750 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:03:11.750 INFO:teuthology.orchestra.run.smithi078.stdout:Total 211 kB/s | 130 kB 00:00 2024-04-24T18:03:11.750 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:03:11.756 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:03:11.756 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:03:11.815 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:03:11.824 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:03:11.992 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:11.994 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:03:12.180 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:03:12.445 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:03:13.918 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 31 kB/s | 4.5 kB 00:00 2024-04-24T18:03:14.597 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 37 kB/s | 4.1 kB 00:00 2024-04-24T18:03:17.152 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: x86_64 2:17.2.7-915.g330067db.el8 ceph 876 k 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout:Installing dependencies: 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: fuse x86_64 2.9.7-17.el8 rhel-8-for-x86_64-baseos-rpms 83 k 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:03:17.153 INFO:teuthology.orchestra.run.smithi049.stdout:Install 2 Packages 2024-04-24T18:03:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:17.154 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 959 k 2024-04-24T18:03:17.154 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 3.1 M 2024-04-24T18:03:17.154 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:03:17.330 INFO:teuthology.orchestra.run.smithi049.stdout:(1/2): fuse-2.9.7-17.el8.x86_64.rpm 474 kB/s | 83 kB 00:00 2024-04-24T18:03:17.578 INFO:teuthology.orchestra.run.smithi049.stdout:(2/2): ceph-fuse-17.2.7-915.g330067db.el8.x86_6 2.0 MB/s | 876 kB 00:00 2024-04-24T18:03:17.579 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:03:17.579 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.2 MB/s | 959 kB 00:00 2024-04-24T18:03:17.591 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:03:17.613 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:03:17.614 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:03:17.704 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:03:17.713 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:03:17.894 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:03:18.191 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : fuse-2.9.7-17.el8.x86_64 1/2 2024-04-24T18:03:18.368 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T18:03:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 2/2 2024-04-24T18:03:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/2 2024-04-24T18:03:23.939 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:03:23.939 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:03:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:24.501 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:03:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:03:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:24.502 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:03:24.675 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install libcephfs2 2024-04-24T18:03:25.034 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:27.231 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-24T18:03:27.909 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 33 kB/s | 4.1 kB 00:00 2024-04-24T18:03:30.023 INFO:teuthology.orchestra.run.smithi078.stdout:Package libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:03:30.218 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:03:30.219 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:03:30.219 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:03:30.351 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install libcephfs-devel 2024-04-24T18:03:30.707 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:33.057 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 24 kB/s | 4.5 kB 00:00 2024-04-24T18:03:33.718 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 35 kB/s | 4.1 kB 00:00 2024-04-24T18:03:35.593 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 2/2 2024-04-24T18:03:35.593 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:03:36.011 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:03:36.011 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 25 k 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 25 k 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 109 k 2024-04-24T18:03:36.012 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:03:36.190 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:36.191 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:03:36.191 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-fuse-2:17.2.7-915.g330067db.el8.x86_64 fuse-2.9.7-17.el8.x86_64 2024-04-24T18:03:36.191 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:36.191 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:03:36.244 INFO:teuthology.orchestra.run.smithi078.stdout:libcephfs-devel-17.2.7-915.g330067db.el8.x86_64 107 kB/s | 25 kB 00:00 2024-04-24T18:03:36.244 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:03:36.244 INFO:teuthology.orchestra.run.smithi078.stdout:Total 107 kB/s | 25 kB 00:00 2024-04-24T18:03:36.244 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:03:36.248 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:03:36.248 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:03:36.250 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:03:36.259 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:03:36.318 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:03:36.360 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install ceph-volume 2024-04-24T18:03:36.511 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:03:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:36.762 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:03:38.816 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 16 kB/s | 4.5 kB 00:00 2024-04-24T18:03:39.502 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-24T18:03:42.042 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repository Size 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume noarch 2:17.2.7-915.g330067db.el8 ceph-noarch 267 k 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:03:42.043 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 267 k 2024-04-24T18:03:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 1.3 M 2024-04-24T18:03:42.044 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:03:42.258 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-volume-17.2.7-915.g330067db.el8.noarch.rpm 1.2 MB/s | 267 kB 00:00 2024-04-24T18:03:42.259 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:03:42.259 INFO:teuthology.orchestra.run.smithi049.stdout:Total 1.2 MB/s | 267 kB 00:00 2024-04-24T18:03:42.259 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:03:42.271 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:03:42.271 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:03:42.301 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:03:42.310 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:03:42.502 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:03:42.748 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:03:43.202 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:03:50.156 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:03:50.156 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:03:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:50.722 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:03:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:03:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:03:50.722 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:03:50.873 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install librados2 2024-04-24T18:03:51.234 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:53.243 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-24T18:03:53.934 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-24T18:03:56.121 INFO:teuthology.orchestra.run.smithi078.stdout:Package librados2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:03:56.308 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:03:56.309 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:03:56.309 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:03:56.461 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install librbd1 2024-04-24T18:03:56.821 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:58.165 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : ceph-volume-2:17.2.7-915.g330067db.el8.noarch 1/1 2024-04-24T18:03:58.165 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:03:58.596 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-24T18:03:58.753 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:58.753 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:03:58.754 INFO:teuthology.orchestra.run.smithi049.stdout: ceph-volume-2:17.2.7-915.g330067db.el8.noarch 2024-04-24T18:03:58.754 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:03:58.754 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:03:58.892 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librados-devel 2024-04-24T18:03:59.274 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:03:59.308 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-04-24T18:04:01.091 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-24T18:04:01.417 INFO:teuthology.orchestra.run.smithi078.stdout:Package librbd1-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:04:01.611 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:04:01.612 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:04:01.612 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:04:01.750 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-rados 2024-04-24T18:04:01.861 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 21 kB/s | 4.1 kB 00:00 2024-04-24T18:04:02.107 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:03.828 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 81 kB/s | 4.5 kB 00:00 2024-04-24T18:04:04.306 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 130 k 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:04:04.307 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 130 k 2024-04-24T18:04:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 464 k 2024-04-24T18:04:04.308 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:04:04.526 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 28 kB/s | 4.1 kB 00:00 2024-04-24T18:04:04.678 INFO:teuthology.orchestra.run.smithi049.stdout:librados-devel-17.2.7-915.g330067db.el8.x86_64. 351 kB/s | 130 kB 00:00 2024-04-24T18:04:04.679 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:04:04.679 INFO:teuthology.orchestra.run.smithi049.stdout:Total 350 kB/s | 130 kB 00:00 2024-04-24T18:04:04.679 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:04:04.686 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:04:04.686 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:04:04.747 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:04:04.756 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:04:04.926 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:04:05.134 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:05.409 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:06.636 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-rados-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:04:06.824 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:04:06.825 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:04:06.826 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:04:06.946 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-rgw 2024-04-24T18:04:07.302 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:11.196 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-24T18:04:11.866 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 34 kB/s | 4.1 kB 00:00 2024-04-24T18:04:13.967 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:04:14.158 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:04:14.159 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:04:14.160 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:04:14.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-cephfs 2024-04-24T18:04:14.656 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:16.670 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-24T18:04:17.372 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-24T18:04:19.468 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:04:19.654 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:04:19.655 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:04:19.655 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:04:19.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install python3-rbd 2024-04-24T18:04:20.151 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:22.093 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : librados-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:22.093 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:04:22.534 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 44 kB/s | 4.5 kB 00:00 2024-04-24T18:04:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:22.684 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:04:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: librados-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:04:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:22.684 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:04:22.858 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install libcephfs2 2024-04-24T18:04:23.238 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:23.239 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-24T18:04:25.083 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 33 kB/s | 4.5 kB 00:00 2024-04-24T18:04:25.330 INFO:teuthology.orchestra.run.smithi078.stdout:Package python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:04:25.517 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:04:25.518 INFO:teuthology.orchestra.run.smithi078.stdout:Nothing to do. 2024-04-24T18:04:25.518 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:04:25.647 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install rbd-fuse 2024-04-24T18:04:25.756 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-04-24T18:04:26.004 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:28.095 INFO:teuthology.orchestra.run.smithi049.stdout:Package libcephfs2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:04:28.274 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-04-24T18:04:28.297 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:04:28.299 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:04:28.299 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:04:28.450 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install libcephfs-devel 2024-04-24T18:04:28.832 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:28.997 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 25 kB/s | 4.1 kB 00:00 2024-04-24T18:04:30.981 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-24T18:04:31.288 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse x86_64 2:17.2.7-915.g330067db.el8 ceph 91 k 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:04:31.289 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:04:31.290 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T18:04:31.290 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:04:31.290 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 91 k 2024-04-24T18:04:31.290 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 268 k 2024-04-24T18:04:31.290 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:04:31.652 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 43 kB/s | 4.1 kB 00:00 2024-04-24T18:04:31.825 INFO:teuthology.orchestra.run.smithi078.stdout:rbd-fuse-17.2.7-915.g330067db.el8.x86_64.rpm 169 kB/s | 91 kB 00:00 2024-04-24T18:04:31.825 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:04:31.825 INFO:teuthology.orchestra.run.smithi078.stdout:Total 169 kB/s | 91 kB 00:00 2024-04-24T18:04:31.826 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:04:31.836 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:04:31.836 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:04:31.893 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:04:31.902 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:04:32.011 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:04:32.199 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:32.457 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:34.200 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:04:34.200 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:04:34.200 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-04-24T18:04:34.200 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:04:34.200 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel x86_64 2:17.2.7-915.g330067db.el8 ceph 25 k 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 25 k 2024-04-24T18:04:34.201 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 109 k 2024-04-24T18:04:34.202 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:04:34.698 INFO:teuthology.orchestra.run.smithi049.stdout:libcephfs-devel-17.2.7-915.g330067db.el8.x86_64 50 kB/s | 25 kB 00:00 2024-04-24T18:04:34.698 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:04:34.698 INFO:teuthology.orchestra.run.smithi049.stdout:Total 50 kB/s | 25 kB 00:00 2024-04-24T18:04:34.699 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:04:34.703 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:04:34.703 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:04:34.705 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:04:34.715 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:04:34.773 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:04:34.937 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:35.231 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:44.903 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:44.903 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:04:45.455 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:04:45.455 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:04:45.455 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:04:45.455 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:04:45.455 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:04:45.631 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install rbd-mirror 2024-04-24T18:04:45.993 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:48.440 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 37 kB/s | 4.5 kB 00:00 2024-04-24T18:04:49.163 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-24T18:04:51.461 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout: Package Arch Version Repo Size 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 3.5 M 2024-04-24T18:04:51.462 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 16 M 2024-04-24T18:04:51.463 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:04:52.244 INFO:teuthology.orchestra.run.smithi078.stdout:rbd-mirror-17.2.7-915.g330067db.el8.x86_64.rpm 4.4 MB/s | 3.5 MB 00:00 2024-04-24T18:04:52.244 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:04:52.245 INFO:teuthology.orchestra.run.smithi078.stdout:Total 4.4 MB/s | 3.5 MB 00:00 2024-04-24T18:04:52.245 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:04:52.257 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:04:52.257 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:04:52.325 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:04:52.334 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:04:52.770 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:04:52.944 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:53.349 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:53.499 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:04:53.499 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:04:54.103 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:54.103 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:04:54.103 INFO:teuthology.orchestra.run.smithi049.stdout: libcephfs-devel-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:04:54.104 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:04:54.104 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:04:54.259 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librados2 2024-04-24T18:04:54.638 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:04:56.745 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-04-24T18:04:57.471 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-24T18:04:59.750 INFO:teuthology.orchestra.run.smithi049.stdout:Package librados2-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:04:59.957 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:04:59.958 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:04:59.958 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:05:00.098 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install librbd1 2024-04-24T18:05:00.476 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:02.382 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 26 kB/s | 4.5 kB 00:00 2024-04-24T18:05:03.106 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-24T18:05:03.165 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:03.165 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:05:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:05:03.725 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:05:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:05:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:05:03.725 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:05:03.932 DEBUG:teuthology.orchestra.run.smithi078:> sudo yum -y install rbd-nbd 2024-04-24T18:05:04.292 INFO:teuthology.orchestra.run.smithi078.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:05.447 INFO:teuthology.orchestra.run.smithi049.stdout:Package librbd1-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:05:05.654 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:05:05.655 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:05:05.656 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:05:05.801 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rados 2024-04-24T18:05:05.993 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 73 kB/s | 4.5 kB 00:00 2024-04-24T18:05:06.179 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:06.645 INFO:teuthology.orchestra.run.smithi078.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 41 kB/s | 4.1 kB 00:00 2024-04-24T18:05:08.122 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 63 kB/s | 4.5 kB 00:00 2024-04-24T18:05:08.825 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-24T18:05:09.038 INFO:teuthology.orchestra.run.smithi078.stdout:Dependencies resolved. 2024-04-24T18:05:09.039 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:05:09.039 INFO:teuthology.orchestra.run.smithi078.stdout: Package Architecture Version Repository Size 2024-04-24T18:05:09.040 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:05:09.040 INFO:teuthology.orchestra.run.smithi078.stdout:Installing: 2024-04-24T18:05:09.040 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd x86_64 2:17.2.7-915.g330067db.el8 ceph 175 k 2024-04-24T18:05:09.040 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:05:09.040 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction Summary 2024-04-24T18:05:09.040 INFO:teuthology.orchestra.run.smithi078.stdout:================================================================================ 2024-04-24T18:05:09.041 INFO:teuthology.orchestra.run.smithi078.stdout:Install 1 Package 2024-04-24T18:05:09.041 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:05:09.041 INFO:teuthology.orchestra.run.smithi078.stdout:Total download size: 175 k 2024-04-24T18:05:09.041 INFO:teuthology.orchestra.run.smithi078.stdout:Installed size: 556 k 2024-04-24T18:05:09.041 INFO:teuthology.orchestra.run.smithi078.stdout:Downloading Packages: 2024-04-24T18:05:09.513 INFO:teuthology.orchestra.run.smithi078.stdout:rbd-nbd-17.2.7-915.g330067db.el8.x86_64.rpm 369 kB/s | 175 kB 00:00 2024-04-24T18:05:09.514 INFO:teuthology.orchestra.run.smithi078.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:05:09.514 INFO:teuthology.orchestra.run.smithi078.stdout:Total 368 kB/s | 175 kB 00:00 2024-04-24T18:05:09.514 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction check 2024-04-24T18:05:09.525 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction check succeeded. 2024-04-24T18:05:09.525 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction test 2024-04-24T18:05:09.581 INFO:teuthology.orchestra.run.smithi078.stdout:Transaction test succeeded. 2024-04-24T18:05:09.590 INFO:teuthology.orchestra.run.smithi078.stdout:Running transaction 2024-04-24T18:05:09.704 INFO:teuthology.orchestra.run.smithi078.stdout: Preparing : 1/1 2024-04-24T18:05:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: Installing : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:10.126 INFO:teuthology.orchestra.run.smithi078.stdout: Running scriptlet: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:11.081 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rados-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:05:11.281 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:05:11.282 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:05:11.282 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:05:11.421 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rgw 2024-04-24T18:05:11.801 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:13.620 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 40 kB/s | 4.5 kB 00:00 2024-04-24T18:05:14.281 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 42 kB/s | 4.1 kB 00:00 2024-04-24T18:05:16.607 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rgw-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:05:16.811 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:05:16.812 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:05:16.812 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:05:16.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-cephfs 2024-04-24T18:05:17.335 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:19.292 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-24T18:05:20.022 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 26 kB/s | 4.1 kB 00:00 2024-04-24T18:05:21.975 INFO:teuthology.orchestra.run.smithi078.stdout: Verifying : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:21.975 INFO:teuthology.orchestra.run.smithi078.stdout:Installed products updated. 2024-04-24T18:05:22.397 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-cephfs-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:05:22.538 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:05:22.538 INFO:teuthology.orchestra.run.smithi078.stdout:Installed: 2024-04-24T18:05:22.538 INFO:teuthology.orchestra.run.smithi078.stdout: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:05:22.538 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:05:22.538 INFO:teuthology.orchestra.run.smithi078.stdout:Complete! 2024-04-24T18:05:22.599 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:05:22.600 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:05:22.600 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:05:22.747 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install python3-rbd 2024-04-24T18:05:22.751 DEBUG:teuthology.parallel:result is None 2024-04-24T18:05:23.128 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:25.036 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 32 kB/s | 4.5 kB 00:00 2024-04-24T18:05:25.694 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 44 kB/s | 4.1 kB 00:00 2024-04-24T18:05:28.047 INFO:teuthology.orchestra.run.smithi049.stdout:Package python3-rbd-2:17.2.7-915.g330067db.el8.x86_64 is already installed. 2024-04-24T18:05:28.254 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:05:28.256 INFO:teuthology.orchestra.run.smithi049.stdout:Nothing to do. 2024-04-24T18:05:28.256 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:05:28.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-fuse 2024-04-24T18:05:28.781 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:30.733 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 29 kB/s | 4.5 kB 00:00 2024-04-24T18:05:31.518 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 19 kB/s | 4.1 kB 00:00 2024-04-24T18:05:34.037 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:05:34.037 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:05:34.037 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-04-24T18:05:34.037 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:05:34.038 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:05:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse x86_64 2:17.2.7-915.g330067db.el8 ceph 91 k 2024-04-24T18:05:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:05:34.038 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:05:34.038 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:05:34.038 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:05:34.038 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:05:34.039 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 91 k 2024-04-24T18:05:34.039 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 268 k 2024-04-24T18:05:34.039 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:05:34.351 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-fuse-17.2.7-915.g330067db.el8.x86_64.rpm 290 kB/s | 91 kB 00:00 2024-04-24T18:05:34.352 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:05:34.352 INFO:teuthology.orchestra.run.smithi049.stdout:Total 289 kB/s | 91 kB 00:00 2024-04-24T18:05:34.352 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:05:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:05:34.364 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:05:34.423 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:05:34.432 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:05:34.591 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:05:34.751 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:35.052 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:46.849 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:05:47.461 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:05:47.461 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:05:47.461 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-fuse-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:05:47.461 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:05:47.461 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:05:47.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-mirror 2024-04-24T18:05:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:05:50.163 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 39 kB/s | 4.5 kB 00:00 2024-04-24T18:05:50.878 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 29 kB/s | 4.1 kB 00:00 2024-04-24T18:05:53.361 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: Package Arch Version Repo Size 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror x86_64 2:17.2.7-915.g330067db.el8 ceph 3.5 M 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:05:53.362 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:05:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:05:53.363 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 3.5 M 2024-04-24T18:05:53.363 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 16 M 2024-04-24T18:05:53.363 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:05:54.871 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-mirror-17.2.7-915.g330067db.el8.x86_64.rpm 2.3 MB/s | 3.5 MB 00:01 2024-04-24T18:05:54.871 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:05:54.872 INFO:teuthology.orchestra.run.smithi049.stdout:Total 2.3 MB/s | 3.5 MB 00:01 2024-04-24T18:05:54.872 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:05:54.886 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:05:54.887 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:05:54.964 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:05:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:05:55.443 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:05:55.617 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:05:56.029 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:06:06.320 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:06:06.320 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:06:06.919 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:06.919 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:06:06.920 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-mirror-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:06:06.920 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:06.920 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:06:07.091 DEBUG:teuthology.orchestra.run.smithi049:> sudo yum -y install rbd-nbd 2024-04-24T18:06:07.488 INFO:teuthology.orchestra.run.smithi049.stdout:Updating Subscription Management repositories. 2024-04-24T18:06:09.458 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - AppStre 30 kB/s | 4.5 kB 00:00 2024-04-24T18:06:10.185 INFO:teuthology.orchestra.run.smithi049.stdout:Red Hat Enterprise Linux 8 for x86_64 - BaseOS 27 kB/s | 4.1 kB 00:00 2024-04-24T18:06:12.724 INFO:teuthology.orchestra.run.smithi049.stdout:Dependencies resolved. 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: Package Architecture Version Repository Size 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout:Installing: 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd x86_64 2:17.2.7-915.g330067db.el8 ceph 175 k 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction Summary 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout:================================================================================ 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout:Install 1 Package 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:12.725 INFO:teuthology.orchestra.run.smithi049.stdout:Total download size: 175 k 2024-04-24T18:06:12.726 INFO:teuthology.orchestra.run.smithi049.stdout:Installed size: 556 k 2024-04-24T18:06:12.726 INFO:teuthology.orchestra.run.smithi049.stdout:Downloading Packages: 2024-04-24T18:06:13.133 INFO:teuthology.orchestra.run.smithi049.stdout:rbd-nbd-17.2.7-915.g330067db.el8.x86_64.rpm 429 kB/s | 175 kB 00:00 2024-04-24T18:06:13.133 INFO:teuthology.orchestra.run.smithi049.stdout:-------------------------------------------------------------------------------- 2024-04-24T18:06:13.133 INFO:teuthology.orchestra.run.smithi049.stdout:Total 428 kB/s | 175 kB 00:00 2024-04-24T18:06:13.134 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction check 2024-04-24T18:06:13.145 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction check succeeded. 2024-04-24T18:06:13.146 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction test 2024-04-24T18:06:13.208 INFO:teuthology.orchestra.run.smithi049.stdout:Transaction test succeeded. 2024-04-24T18:06:13.217 INFO:teuthology.orchestra.run.smithi049.stdout:Running transaction 2024-04-24T18:06:13.343 INFO:teuthology.orchestra.run.smithi049.stdout: Preparing : 1/1 2024-04-24T18:06:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: Installing : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:06:13.809 INFO:teuthology.orchestra.run.smithi049.stdout: Running scriptlet: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:06:27.173 INFO:teuthology.orchestra.run.smithi049.stdout: Verifying : rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 1/1 2024-04-24T18:06:27.173 INFO:teuthology.orchestra.run.smithi049.stdout:Installed products updated. 2024-04-24T18:06:27.778 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:27.778 INFO:teuthology.orchestra.run.smithi049.stdout:Installed: 2024-04-24T18:06:27.778 INFO:teuthology.orchestra.run.smithi049.stdout: rbd-nbd-2:17.2.7-915.g330067db.el8.x86_64 2024-04-24T18:06:27.778 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:27.778 INFO:teuthology.orchestra.run.smithi049.stdout:Complete! 2024-04-24T18:06:27.983 DEBUG:teuthology.parallel:result is None 2024-04-24T18:06:27.984 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T18:06:28.129 DEBUG:teuthology.orchestra.run.smithi049:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T18:06:28.152 INFO:teuthology.orchestra.run.smithi049.stdout:17.2.7-915.g330067db.el8 2024-04-24T18:06:28.153 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-915.g330067db.el8 2024-04-24T18:06:28.153 INFO:teuthology.task.install:The correct ceph version 17.2.7-915.g330067db is installed. 2024-04-24T18:06:28.155 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=330067dbce47655d1f762ef14d546d2fde16a6a9 2024-04-24T18:06:28.424 DEBUG:teuthology.orchestra.run.smithi078:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-24T18:06:28.453 INFO:teuthology.orchestra.run.smithi078.stdout:17.2.7-915.g330067db.el8 2024-04-24T18:06:28.453 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-915.g330067db.el8 2024-04-24T18:06:28.453 INFO:teuthology.task.install:The correct ceph version 17.2.7-915.g330067db is installed. 2024-04-24T18:06:28.455 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-24T18:06:28.455 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:28.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T18:06:28.486 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:06:28.486 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-24T18:06:28.540 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-24T18:06:28.540 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:28.540 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T18:06:28.569 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T18:06:28.638 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:06:28.638 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/daemon-helper 2024-04-24T18:06:28.668 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-24T18:06:28.738 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-24T18:06:28.738 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:28.738 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T18:06:28.766 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T18:06:28.835 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:06:28.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-24T18:06:28.864 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-24T18:06:28.934 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-24T18:06:28.934 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:28.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T18:06:28.962 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T18:06:29.030 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:06:29.030 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/usr/bin/stdin-killer 2024-04-24T18:06:29.058 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-24T18:06:29.125 INFO:teuthology.run_tasks:Running task ceph... 2024-04-24T18:06:29.218 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-24T18:06:29.218 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 777 /var/log/ceph 2024-04-24T18:06:29.220 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 777 /var/log/ceph 2024-04-24T18:06:29.247 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-24T18:06:29.247 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T18:06:29.289 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-24T18:06:29.315 INFO:tasks.ceph:Creating extra log directories... 2024-04-24T18:06:29.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T18:06:29.358 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-24T18:06:29.396 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-24T18:06:29.396 INFO:tasks.ceph:config {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-24T18:06:29.397 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641', 'branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'description': 'rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writearound}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7671641', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'machine': {'cpus': 1, 'disk': 40, 'ram': 8000}, 'volumes': {'count': 4, 'size': 30}}], 'os_type': 'rhel', 'os_version': '8.6', 'overrides': {'admin_socket': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 99, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 9954, 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rbd', 'suite_branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_330067dbce47655d1f762ef14d546d2fde16a6a9/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9', 'targets': {'smithi049.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBAmpKeVOaV+kk9DAKzd4G75t9zj75Y7RbtC5dbT4qkkJnSiDmlEp5teDAKuw9Y2Pahlg5fVaJUzSKw+9SIOVJkg=', 'smithi078.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLfZwGy4KggpIQ5Xr3FpzGaV92di5pOwGC7vllv/74eSJo0Lobhx/D49PoG7Ya/JvdGWv84/cQGpAO67zRImDz4='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'exec': {'client.0': ['sudo ceph osd pool create cache 4', 'sudo ceph osd tier add rbd cache', 'sudo ceph osd tier cache-mode cache writeback', 'sudo ceph osd tier set-overlay rbd cache', 'sudo ceph osd pool set cache hit_set_type bloom', 'sudo ceph osd pool set cache hit_set_count 8', 'sudo ceph osd pool set cache hit_set_period 60', 'sudo ceph osd pool set cache target_max_objects 250']}}, {'thrashosds': {'timeout': 1200}}, {'rbd_fsx': {'clients': ['client.0'], 'ops': 6000}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-24_14:09:24', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-24T18:06:29.397 DEBUG:teuthology.orchestra.run.smithi049:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T18:06:29.427 DEBUG:teuthology.orchestra.run.smithi078:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-24T18:06:29.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T18:06:29.483 DEBUG:teuthology.orchestra.run.smithi078:> sudo install -d -m0777 -- /var/run/ceph 2024-04-24T18:06:29.510 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:29.510 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T18:06:29.567 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T18:06:29.568 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_1 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 182516 Links: 1 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-24 18:05:55.793942124 +0000 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-24 17:56:13.000279880 +0000 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-24 17:56:13.000279880 +0000 2024-04-24T18:06:29.625 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-04-24T18:06:29.626 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T18:06:29.692 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-24T18:06:29.692 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-24T18:06:29.692 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 9.5909e-05 s, 5.3 MB/s 2024-04-24T18:06:29.693 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T18:06:29.752 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_2 2024-04-24T18:06:29.810 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T18:06:29.810 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:29.810 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 179861 Links: 1 2024-04-24T18:06:29.810 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:29.811 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:29.811 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-24 18:05:55.794942097 +0000 2024-04-24T18:06:29.811 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-24 17:56:13.330270791 +0000 2024-04-24T18:06:29.811 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-24 17:56:13.330270791 +0000 2024-04-24T18:06:29.811 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-04-24T18:06:29.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T18:06:29.876 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-24T18:06:29.876 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-24T18:06:29.876 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000105888 s, 4.8 MB/s 2024-04-24T18:06:29.877 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T18:06:29.935 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_3 2024-04-24T18:06:29.993 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T18:06:29.993 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:29.993 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 181649 Links: 1 2024-04-24T18:06:29.993 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:29.993 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:29.993 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-24 18:05:55.794942097 +0000 2024-04-24T18:06:29.993 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-24 17:56:13.682261095 +0000 2024-04-24T18:06:29.994 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-24 17:56:13.682261095 +0000 2024-04-24T18:06:29.994 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-04-24T18:06:29.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T18:06:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-24T18:06:30.059 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-24T18:06:30.060 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 8.7515e-05 s, 5.9 MB/s 2024-04-24T18:06:30.060 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T18:06:30.119 DEBUG:teuthology.orchestra.run.smithi049:> stat /dev/vg_nvme/lv_4 2024-04-24T18:06:30.176 INFO:teuthology.orchestra.run.smithi049.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout:Device: 6h/6d Inode: 181724 Links: 1 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout:Access: 2024-04-24 18:05:55.794942097 +0000 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout:Modify: 2024-04-24 17:56:14.040251234 +0000 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout:Change: 2024-04-24 17:56:14.040251234 +0000 2024-04-24T18:06:30.177 INFO:teuthology.orchestra.run.smithi049.stdout: Birth: - 2024-04-24T18:06:30.178 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T18:06:30.242 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records in 2024-04-24T18:06:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:1+0 records out 2024-04-24T18:06:30.243 INFO:teuthology.orchestra.run.smithi049.stderr:512 bytes copied, 0.000110815 s, 4.6 MB/s 2024-04-24T18:06:30.244 DEBUG:teuthology.orchestra.run.smithi049:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T18:06:30.301 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:30.302 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:06:30.302 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/scratch_devs of=/dev/stdout 2024-04-24T18:06:30.321 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-24T18:06:30.321 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_1 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 178493 Links: 1 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-24 18:04:53.122800998 +0000 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-24 17:55:41.907271881 +0000 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-24 17:55:41.907271881 +0000 2024-04-24T18:06:30.381 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2024-04-24T18:06:30.382 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-24T18:06:30.450 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-24T18:06:30.450 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-24T18:06:30.450 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000273746 s, 1.9 MB/s 2024-04-24T18:06:30.451 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-24T18:06:30.510 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_2 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 185346 Links: 1 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-24 18:04:53.122800998 +0000 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-24 17:55:42.254272563 +0000 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-24 17:55:42.254272563 +0000 2024-04-24T18:06:30.570 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2024-04-24T18:06:30.571 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-24T18:06:30.637 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-24T18:06:30.637 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-24T18:06:30.637 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000222002 s, 2.3 MB/s 2024-04-24T18:06:30.638 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-24T18:06:30.697 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_3 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 180662 Links: 1 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-24 18:04:53.122800998 +0000 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-24 17:55:42.583273208 +0000 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-24 17:55:42.583273208 +0000 2024-04-24T18:06:30.756 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2024-04-24T18:06:30.757 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-24T18:06:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-24T18:06:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-24T18:06:30.822 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000170749 s, 3.0 MB/s 2024-04-24T18:06:30.823 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-24T18:06:30.881 DEBUG:teuthology.orchestra.run.smithi078:> stat /dev/vg_nvme/lv_4 2024-04-24T18:06:30.938 INFO:teuthology.orchestra.run.smithi078.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-24T18:06:30.938 INFO:teuthology.orchestra.run.smithi078.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-24T18:06:30.938 INFO:teuthology.orchestra.run.smithi078.stdout:Device: 6h/6d Inode: 183721 Links: 1 2024-04-24T18:06:30.938 INFO:teuthology.orchestra.run.smithi078.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-24T18:06:30.938 INFO:teuthology.orchestra.run.smithi078.stdout:Context: system_u:object_r:device_t:s0 2024-04-24T18:06:30.939 INFO:teuthology.orchestra.run.smithi078.stdout:Access: 2024-04-24 18:04:53.122800998 +0000 2024-04-24T18:06:30.939 INFO:teuthology.orchestra.run.smithi078.stdout:Modify: 2024-04-24 17:55:42.941273911 +0000 2024-04-24T18:06:30.939 INFO:teuthology.orchestra.run.smithi078.stdout:Change: 2024-04-24 17:55:42.941273911 +0000 2024-04-24T18:06:30.939 INFO:teuthology.orchestra.run.smithi078.stdout: Birth: - 2024-04-24T18:06:30.939 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-24T18:06:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records in 2024-04-24T18:06:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:1+0 records out 2024-04-24T18:06:31.004 INFO:teuthology.orchestra.run.smithi078.stderr:512 bytes copied, 0.000231348 s, 2.2 MB/s 2024-04-24T18:06:31.005 DEBUG:teuthology.orchestra.run.smithi078:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-24T18:06:31.063 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:31.063 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T18:06:31.064 INFO:tasks.ceph:Generating config... 2024-04-24T18:06:31.065 INFO:tasks.ceph:[client] rbd cache = True 2024-04-24T18:06:31.065 INFO:tasks.ceph:[client] rbd cache policy = writearound 2024-04-24T18:06:31.065 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-04-24T18:06:31.065 INFO:tasks.ceph:[global] mon warn on pool no app = False 2024-04-24T18:06:31.065 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-04-24T18:06:31.065 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-24T18:06:31.065 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-24T18:06:31.065 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-24T18:06:31.065 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-24T18:06:31.065 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-24T18:06:31.065 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-24T18:06:31.065 INFO:tasks.ceph:[osd] bluestore compression algorithm = lz4 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] bluestore compression mode = aggressive 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-24T18:06:31.066 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-24T18:06:31.067 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-04-24T18:06:31.067 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-24T18:06:31.067 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-04-24T18:06:31.067 INFO:tasks.ceph:Setting up mon.a... 2024-04-24T18:06:31.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-24T18:06:31.139 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.keyring 2024-04-24T18:06:31.142 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-24T18:06:31.188 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T18:06:31.217 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.49'), ('mon.c', '[v2:172.21.15.49:3301,v1:172.21.15.49:6790]'), ('mon.b', '172.21.15.78')] 2024-04-24T18:06:31.217 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.49,[v2:172.21.15.49:3301,v1:172.21.15.49:6790],172.21.15.78', 'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'rbd cache': True, 'rbd cache policy': 'writearound'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-04-24T18:06:31.218 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:31.218 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T18:06:31.275 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.49 --addv c '[v2:172.21.15.49:3301,v1:172.21.15.49:6790]' --add b 172.21.15.78 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: generated fsid 5dae32d6-9101-4f7b-a6ef-ca5186fee71b 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:setting min_mon_release = octopus 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 0 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:fsid 5dae32d6-9101-4f7b-a6ef-ca5186fee71b 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:last_changed 2024-04-24T18:06:31.371639+0000 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-04-24T18:06:31.371639+0000 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:min_mon_release 15 (octopus) 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:election_strategy: 1 2024-04-24T18:06:31.363 INFO:teuthology.orchestra.run.smithi049.stdout:0: [v2:172.21.15.49:3300/0,v1:172.21.15.49:6789/0] mon.a 2024-04-24T18:06:31.364 INFO:teuthology.orchestra.run.smithi049.stdout:1: [v2:172.21.15.78:3300/0,v1:172.21.15.78:6789/0] mon.b 2024-04-24T18:06:31.364 INFO:teuthology.orchestra.run.smithi049.stdout:2: [v2:172.21.15.49:3301/0,v1:172.21.15.49:6790/0] mon.c 2024-04-24T18:06:31.364 INFO:teuthology.orchestra.run.smithi049.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-04-24T18:06:31.365 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-24T18:06:31.382 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 5dae32d6-9101-4f7b-a6ef-ca5186fee71b... 2024-04-24T18:06:31.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T18:06:31.424 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-24T18:06:31.469 INFO:teuthology.orchestra.run.smithi049.stdout:[global] 2024-04-24T18:06:31.469 INFO:teuthology.orchestra.run.smithi049.stdout: chdir = "" 2024-04-24T18:06:31.469 INFO:teuthology.orchestra.run.smithi049.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T18:06:31.469 INFO:teuthology.orchestra.run.smithi049.stdout: auth supported = cephx 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: filestore xattr use omap = true 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: mon clock drift allowed = 1.000 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: osd crush chooseleaf type = 0 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: auth debug = true 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on old message = true 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: ms die on bug = true 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: mon pg warn max object skew = 0 2024-04-24T18:06:31.470 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default size = 2 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow primary affinity = true 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd allow pg remap = true 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on legacy crush tunables = false 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on crush straw calc version zero = false 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on no sortbitwise = false 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on osd down out interval zero = false 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on too few osds = false 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T18:06:31.471 INFO:teuthology.orchestra.run.smithi049.stdout: mon_allow_pool_size_one = true 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: osd default data pool replay window = 5 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: mon allow pool delete = true 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: mon cluster log file level = debug 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: debug asserts on shutdown = true 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: mon health detail to clog = false 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: mon host = "172.21.15.49,[v2:172.21.15.49:3301,v1:172.21.15.49:6790],172.21.15.78" 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: mon client directed command retry = 5 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: mon warn on pool no app = False 2024-04-24T18:06:31.472 INFO:teuthology.orchestra.run.smithi049.stdout: ms inject socket failures = 5000 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: fsid = 5dae32d6-9101-4f7b-a6ef-ca5186fee71b 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout:[osd] 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd journal size = 100 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub load threshold = 5.0 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd scrub max interval = 600 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd mclock profile = high_recovery_ops 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd recover clone overlap = true 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd recovery max chunk = 1048576 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug shutdown = true 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug op order = true 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug verify stray on activate = true 2024-04-24T18:06:31.473 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: osd open classes on start = true 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug pg log writeout = true 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: osd deep scrub update digest min age = 30 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: osd map max advance = 10 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: journal zero on create = true 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: filestore ondisk finisher threads = 3 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: filestore apply finisher threads = 3 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: bdev debug aio = true 2024-04-24T18:06:31.474 INFO:teuthology.orchestra.run.smithi049.stdout: osd debug misdirected ops = true 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore block size = 96636764160 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore compression algorithm = lz4 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore compression mode = aggressive 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: bluestore fsck on mount = True 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluefs = 1/20 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: debug bluestore = 1/20 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: debug osd = 20 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: debug rocksdb = 4/10 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd full ratio = 0.9 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd nearfull ratio = 0.8 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: osd failsafe full ratio = 0.95 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: osd max markdown count = 1000 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: osd objectstore = bluestore 2024-04-24T18:06:31.475 INFO:teuthology.orchestra.run.smithi049.stdout: osd shutdown pgref assert = True 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr] 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: debug mgr = 20 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: mgr/telemetry/nag = false 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout:[mon] 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: debug ms = 1 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: debug mon = 20 2024-04-24T18:06:31.476 INFO:teuthology.orchestra.run.smithi049.stdout: debug paxos = 20 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: debug auth = 20 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon data avail warn = 5 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon mgr mkfs grace = 240 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min pgs per osd = 4 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd reporter subtree level = osd 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon osd prime pg temp = true 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon reweight min bytes per osd = 10 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T18:06:31.477 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: # 1m isn't quite enough 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: mon_down_mkfs_grace = 2m 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: mon_warn_on_filestore_osds = false 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout:[client] 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: rgw cache enabled = true 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable ops log = true 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: rgw enable usage log = true 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: rbd cache = True 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout: rbd cache policy = writearound 2024-04-24T18:06:31.478 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.a] 2024-04-24T18:06:31.479 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.c] 2024-04-24T18:06:31.479 INFO:teuthology.orchestra.run.smithi049.stdout:[mon.b] 2024-04-24T18:06:31.479 INFO:teuthology.orchestra.run.smithi078.stdout:[global] 2024-04-24T18:06:31.479 INFO:teuthology.orchestra.run.smithi078.stdout: chdir = "" 2024-04-24T18:06:31.479 INFO:teuthology.orchestra.run.smithi078.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-24T18:06:31.479 INFO:teuthology.orchestra.run.smithi078.stdout: auth supported = cephx 2024-04-24T18:06:31.479 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: filestore xattr use omap = true 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: mon clock drift allowed = 1.000 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: osd crush chooseleaf type = 0 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: auth debug = true 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on old message = true 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: ms die on bug = true 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: mon pg warn max object skew = 0 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: # disable pg_autoscaler by default for new pools 2024-04-24T18:06:31.480 INFO:teuthology.orchestra.run.smithi078.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default size = 2 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow primary affinity = true 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd allow pg remap = true 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on legacy crush tunables = false 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on crush straw calc version zero = false 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on no sortbitwise = false 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on osd down out interval zero = false 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on too few osds = false 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: mon_allow_pool_size_one = true 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.481 INFO:teuthology.orchestra.run.smithi078.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: osd default data pool replay window = 5 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: mon allow pool delete = true 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: mon cluster log file level = debug 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: debug asserts on shutdown = true 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: mon health detail to clog = false 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: mon host = "172.21.15.49,[v2:172.21.15.49:3301,v1:172.21.15.49:6790],172.21.15.78" 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: mon client directed command retry = 5 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: mon warn on pool no app = False 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: ms inject socket failures = 5000 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: fsid = 5dae32d6-9101-4f7b-a6ef-ca5186fee71b 2024-04-24T18:06:31.482 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout:[osd] 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd journal size = 100 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub load threshold = 5.0 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd scrub max interval = 600 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd mclock profile = high_recovery_ops 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd recover clone overlap = true 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd recovery max chunk = 1048576 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug shutdown = true 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug op order = true 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug verify stray on activate = true 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd open classes on start = true 2024-04-24T18:06:31.483 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug pg log writeout = true 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: osd deep scrub update digest min age = 30 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: osd map max advance = 10 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: journal zero on create = true 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: filestore ondisk finisher threads = 3 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: filestore apply finisher threads = 3 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: bdev debug aio = true 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: osd debug misdirected ops = true 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore block size = 96636764160 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore compression algorithm = lz4 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore compression mode = aggressive 2024-04-24T18:06:31.484 INFO:teuthology.orchestra.run.smithi078.stdout: bluestore fsck on mount = True 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluefs = 1/20 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: debug bluestore = 1/20 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: debug osd = 20 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: debug rocksdb = 4/10 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd backfillfull_ratio = 0.85 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd full ratio = 0.9 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd nearfull ratio = 0.8 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: osd failsafe full ratio = 0.95 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: osd max markdown count = 1000 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: osd objectstore = bluestore 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: osd shutdown pgref assert = True 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr] 2024-04-24T18:06:31.485 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: debug mgr = 20 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: mgr/telemetry/nag = false 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout:[mon] 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: debug ms = 1 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: debug mon = 20 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: debug paxos = 20 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: debug auth = 20 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: mon data avail warn = 5 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: mon mgr mkfs grace = 240 2024-04-24T18:06:31.486 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min pgs per osd = 4 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd reporter subtree level = osd 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: mon osd prime pg temp = true 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: mon reweight min bytes per osd = 10 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: auth mon ticket ttl = 660 # 11m 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: auth service ticket ttl = 240 # 4m 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: # don't complain about insecure global_id in the test suite 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: # 1m isn't quite enough 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: mon_down_mkfs_grace = 2m 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.487 INFO:teuthology.orchestra.run.smithi078.stdout: mon_warn_on_filestore_osds = false 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout:[client] 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: rgw cache enabled = true 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable ops log = true 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: rgw enable usage log = true 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: rbd cache = True 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout: rbd cache policy = writearound 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.a] 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.c] 2024-04-24T18:06:31.488 INFO:teuthology.orchestra.run.smithi078.stdout:[mon.b] 2024-04-24T18:06:31.490 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-24T18:06:31.490 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-24T18:06:31.586 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-24T18:06:31.586 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:31.586 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-24T18:06:31.603 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:31.603 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-24T18:06:31.660 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:06:31.660 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:31.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T18:06:31.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T18:06:31.738 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:06:31.738 DEBUG:teuthology.orchestra.run.smithi049:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:31.795 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:06:31.795 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:06:31.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-24T18:06:31.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-24T18:06:31.833 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:06:31.833 DEBUG:teuthology.orchestra.run.smithi078:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:31.889 INFO:tasks.ceph:Setting up mon nodes... 2024-04-24T18:06:31.890 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-24T18:06:31.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T18:06:31.942 INFO:teuthology.orchestra.run.smithi049.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-04-24T18:06:31.945 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T18:06:32.013 INFO:teuthology.orchestra.run.smithi078.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-24T18:06:32.017 INFO:tasks.ceph:Setting up mds nodes... 2024-04-24T18:06:32.017 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-24T18:06:32.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-24T18:06:32.059 INFO:teuthology.orchestra.run.smithi049.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-24T18:06:32.072 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-04-24T18:06:32.112 INFO:teuthology.orchestra.run.smithi078.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-04-24T18:06:32.125 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-24T18:06:32.125 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi049.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi078.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-04-24T18:06:32.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-24T18:06:32.153 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:32.153 INFO:tasks.ceph:role: osd.0 2024-04-24T18:06:32.153 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:06:32.153 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T18:06:32.224 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:32.224 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:32.225 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:32.225 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:32.225 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:32.246 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-24T18:06:32.248 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-24T18:06:32.248 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-24T18:06:32.329 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-24T18:06:32.402 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-24T18:06:32.470 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:32.470 INFO:tasks.ceph:role: osd.1 2024-04-24T18:06:32.470 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:06:32.471 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:32.538 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:32.561 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-24T18:06:32.562 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-24T18:06:32.562 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-04-24T18:06:32.645 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-24T18:06:32.717 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-24T18:06:32.784 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:32.784 INFO:tasks.ceph:role: osd.2 2024-04-24T18:06:32.784 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:06:32.784 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T18:06:32.851 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:32.852 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:32.852 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:32.852 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:32.852 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:32.873 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-24T18:06:32.875 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-24T18:06:32.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-04-24T18:06:32.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-24T18:06:33.027 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-24T18:06:33.095 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:33.095 INFO:tasks.ceph:role: osd.3 2024-04-24T18:06:33.095 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:06:33.095 DEBUG:teuthology.orchestra.run.smithi049:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:33.162 INFO:teuthology.orchestra.run.smithi049.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:33.184 INFO:teuthology.orchestra.run.smithi049.stdout:Discarding blocks...Done. 2024-04-24T18:06:33.186 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi049.front.sepia.ceph.com -o noatime 2024-04-24T18:06:33.186 DEBUG:teuthology.orchestra.run.smithi049:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-04-24T18:06:33.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-24T18:06:33.338 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:33.430+0000 7f9eabb8d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-24T18:06:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:33.430+0000 7f9eabb8d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-24T18:06:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:33.430+0000 7f9eabb8d540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T18:06:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:33.430+0000 7f9eabb8d540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T18:06:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:33.430+0000 7f9eabb8d540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-24T18:06:33.423 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:33.431+0000 7f9eabb8d540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-24T18:06:33.945 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:33.952+0000 7f9eabb8d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:34.706 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:34.714+0000 7f9eabb8d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:35.471 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:35.478+0000 7f9eabb8d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:36.268 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-24T18:06:36.302 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:36.391 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:36.398+0000 7f4e2dbfe540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-24T18:06:36.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:36.399+0000 7f4e2dbfe540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-24T18:06:36.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:36.399+0000 7f4e2dbfe540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T18:06:36.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:36.399+0000 7f4e2dbfe540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T18:06:36.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:36.399+0000 7f4e2dbfe540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-24T18:06:36.392 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:36.399+0000 7f4e2dbfe540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-24T18:06:36.915 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:36.923+0000 7f4e2dbfe540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:37.681 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:37.688+0000 7f4e2dbfe540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:38.450 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:38.458+0000 7f4e2dbfe540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:39.244 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-24T18:06:39.277 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:39.367 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:39.374+0000 7fbd2af4a540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-24T18:06:39.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:39.375+0000 7fbd2af4a540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-24T18:06:39.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:39.375+0000 7fbd2af4a540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T18:06:39.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:39.375+0000 7fbd2af4a540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T18:06:39.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:39.375+0000 7fbd2af4a540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-24T18:06:39.368 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:39.375+0000 7fbd2af4a540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-24T18:06:39.894 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:39.901+0000 7fbd2af4a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:40.660 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:40.667+0000 7fbd2af4a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:41.422 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:41.429+0000 7fbd2af4a540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:42.217 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-24T18:06:42.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:42.341 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:42.348+0000 7fb493fc2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-24T18:06:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:42.348+0000 7fb493fc2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-24T18:06:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:42.348+0000 7fb493fc2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T18:06:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:42.348+0000 7fb493fc2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T18:06:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:42.348+0000 7fb493fc2540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-24T18:06:42.342 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:42.349+0000 7fb493fc2540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-24T18:06:42.866 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:42.873+0000 7fb493fc2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:43.627 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:43.635+0000 7fb493fc2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:44.390 INFO:teuthology.orchestra.run.smithi049.stderr:2024-04-24T18:06:44.398+0000 7fb493fc2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:45.184 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-24T18:06:45.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-04-24T18:06:45.252 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:45.252 INFO:tasks.ceph:role: osd.4 2024-04-24T18:06:45.252 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:06:45.252 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-24T18:06:45.333 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:45.334 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:45.771 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-04-24T18:06:45.773 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-04-24T18:06:45.773 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-04-24T18:06:45.944 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-04-24T18:06:45.985 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-04-24T18:06:46.058 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:46.058 INFO:tasks.ceph:role: osd.5 2024-04-24T18:06:46.058 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:06:46.058 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:46.138 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:46.139 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:46.563 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-04-24T18:06:46.565 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-04-24T18:06:46.565 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-04-24T18:06:46.676 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-04-24T18:06:46.714 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-04-24T18:06:46.786 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:46.786 INFO:tasks.ceph:role: osd.6 2024-04-24T18:06:46.786 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:06:46.786 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:46.867 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:47.325 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-04-24T18:06:47.327 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-04-24T18:06:47.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-04-24T18:06:47.467 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-04-24T18:06:47.505 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-04-24T18:06:47.575 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-04-24T18:06:47.576 INFO:tasks.ceph:role: osd.7 2024-04-24T18:06:47.576 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:06:47.576 DEBUG:teuthology.orchestra.run.smithi078:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-04-24T18:06:47.653 INFO:teuthology.orchestra.run.smithi078.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-04-24T18:06:47.653 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-24T18:06:47.653 INFO:teuthology.orchestra.run.smithi078.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-24T18:06:47.653 INFO:teuthology.orchestra.run.smithi078.stdout: = reflink=1 bigtime=0 inobtcount=0 2024-04-24T18:06:47.654 INFO:teuthology.orchestra.run.smithi078.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-24T18:06:47.654 INFO:teuthology.orchestra.run.smithi078.stdout: = sunit=0 swidth=0 blks 2024-04-24T18:06:47.654 INFO:teuthology.orchestra.run.smithi078.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-24T18:06:47.654 INFO:teuthology.orchestra.run.smithi078.stdout:log =internal log bsize=4096 blocks=11443, version=2 2024-04-24T18:06:47.654 INFO:teuthology.orchestra.run.smithi078.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-24T18:06:47.654 INFO:teuthology.orchestra.run.smithi078.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-24T18:06:48.090 INFO:teuthology.orchestra.run.smithi078.stdout:Discarding blocks...Done. 2024-04-24T18:06:48.092 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi078.front.sepia.ceph.com -o noatime 2024-04-24T18:06:48.092 DEBUG:teuthology.orchestra.run.smithi078:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-04-24T18:06:48.260 DEBUG:teuthology.orchestra.run.smithi078:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-04-24T18:06:48.299 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:48.399+0000 7fef85d42540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-04-24T18:06:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:48.399+0000 7fef85d42540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-04-24T18:06:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:48.399+0000 7fef85d42540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T18:06:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:48.399+0000 7fef85d42540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T18:06:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:48.399+0000 7fef85d42540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-04-24T18:06:48.392 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:48.400+0000 7fef85d42540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-04-24T18:06:48.917 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:48.925+0000 7fef85d42540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:49.678 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:49.686+0000 7fef85d42540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:50.440 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:50.448+0000 7fef85d42540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:51.233 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-04-24T18:06:51.268 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:51.359 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:51.367+0000 7ff70dac3540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-04-24T18:06:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:51.367+0000 7ff70dac3540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-04-24T18:06:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:51.367+0000 7ff70dac3540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T18:06:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:51.367+0000 7ff70dac3540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T18:06:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:51.367+0000 7ff70dac3540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-04-24T18:06:51.360 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:51.368+0000 7ff70dac3540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-04-24T18:06:51.887 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:51.895+0000 7ff70dac3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:52.654 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:52.662+0000 7ff70dac3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:53.416 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:53.424+0000 7ff70dac3540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:54.210 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-04-24T18:06:54.243 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:54.342+0000 7f782b46c540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-04-24T18:06:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:54.342+0000 7f782b46c540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-04-24T18:06:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:54.342+0000 7f782b46c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T18:06:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:54.342+0000 7f782b46c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T18:06:54.334 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:54.342+0000 7f782b46c540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-04-24T18:06:54.335 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:54.342+0000 7f782b46c540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-04-24T18:06:54.864 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:54.872+0000 7f782b46c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:55.629 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:55.636+0000 7f782b46c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:56.390 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:56.398+0000 7f782b46c540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:57.183 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-04-24T18:06:57.216 DEBUG:teuthology.orchestra.run.smithi078:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:06:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:57.313+0000 7fe66eb4d540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-04-24T18:06:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:57.313+0000 7fe66eb4d540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-04-24T18:06:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:57.313+0000 7fe66eb4d540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T18:06:57.305 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:57.313+0000 7fe66eb4d540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T18:06:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:57.313+0000 7fe66eb4d540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-04-24T18:06:57.306 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:57.314+0000 7fe66eb4d540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-04-24T18:06:57.829 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:57.837+0000 7fe66eb4d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:58.595 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:58.603+0000 7fe66eb4d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:06:59.357 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:06:59.365+0000 7fe66eb4d540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2024-04-24T18:07:00.154 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-04-24T18:07:00.187 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-24T18:07:00.188 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:00.188 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-04-24T18:07:00.221 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:00.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-24T18:07:00.294 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:00.294 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-24T18:07:00.365 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:00.365 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-24T18:07:00.436 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:00.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-24T18:07:00.505 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:00.505 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-24T18:07:00.539 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:00.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-04-24T18:07:00.611 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:00.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-04-24T18:07:00.684 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:00.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-04-24T18:07:00.756 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:00.756 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-04-24T18:07:00.828 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:00.828 DEBUG:teuthology.orchestra.run.smithi049:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-24T18:07:00.849 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:00.849 DEBUG:teuthology.orchestra.run.smithi078:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-04-24T18:07:00.887 INFO:tasks.ceph:Adding keys to all mons... 2024-04-24T18:07:00.887 DEBUG:teuthology.orchestra.run.smithi049:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T18:07:00.891 DEBUG:teuthology.orchestra.run.smithi078:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-24T18:07:00.930 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.y] 2024-04-24T18:07:00.930 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAnSilmlHqsOBAAtukHzVPm/7b+jMOlMAkgOA== 2024-04-24T18:07:00.930 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.0] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQApSilme7O7GRAAnkcx54m7tvuDjB/uIxQiHQ== 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.1] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAsSilmWx7aFxAAiT2F3AOVonYCxdbI97/9aw== 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.2] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAvSilmCaJpFhAAmwo3QUon7BCqeR3DsqgpZA== 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.3] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAySilmJR/bFBAA6LapiubtK+1MiYq5wil2PQ== 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout:[mgr.x] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAoSilmsuZWARAASPw/o/zf7KnzZJBh+B2KUA== 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.4] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQA4Silma7XaFxAAfjMs1VsrT0PFPU5KyXAKAQ== 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.5] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQA7SilmKyjzFRAAtBXcADP4l4kFTaLbNmVrtQ== 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.6] 2024-04-24T18:07:00.931 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQA+Silm9ettFBAAWGAn17qBTx6ccOgHYblCgA== 2024-04-24T18:07:00.932 INFO:teuthology.orchestra.run.smithi049.stdout:[osd.7] 2024-04-24T18:07:00.932 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQBBSilmwfC2EhAA3rdWZvkYT+wPhbLCOVtWjw== 2024-04-24T18:07:00.932 INFO:teuthology.orchestra.run.smithi049.stdout:[client.0] 2024-04-24T18:07:00.932 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAoSilmqYUOBBAAXVI0rrfOLOUnxTb+zz9b2w== 2024-04-24T18:07:00.932 INFO:teuthology.orchestra.run.smithi049.stdout:[client.1] 2024-04-24T18:07:00.932 INFO:teuthology.orchestra.run.smithi049.stdout: key = AQAoSilmqTc4BxAAI/hXgahrCmH6RLXgGsMpow== 2024-04-24T18:07:00.955 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.y] 2024-04-24T18:07:00.955 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAnSilmlHqsOBAAtukHzVPm/7b+jMOlMAkgOA== 2024-04-24T18:07:00.955 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.0] 2024-04-24T18:07:00.955 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQApSilme7O7GRAAnkcx54m7tvuDjB/uIxQiHQ== 2024-04-24T18:07:00.955 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.1] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAsSilmWx7aFxAAiT2F3AOVonYCxdbI97/9aw== 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.2] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAvSilmCaJpFhAAmwo3QUon7BCqeR3DsqgpZA== 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.3] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAySilmJR/bFBAA6LapiubtK+1MiYq5wil2PQ== 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout:[mgr.x] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAoSilmsuZWARAASPw/o/zf7KnzZJBh+B2KUA== 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.4] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQA4Silma7XaFxAAfjMs1VsrT0PFPU5KyXAKAQ== 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.5] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQA7SilmKyjzFRAAtBXcADP4l4kFTaLbNmVrtQ== 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.6] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQA+Silm9ettFBAAWGAn17qBTx6ccOgHYblCgA== 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout:[osd.7] 2024-04-24T18:07:00.956 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQBBSilmwfC2EhAA3rdWZvkYT+wPhbLCOVtWjw== 2024-04-24T18:07:00.957 INFO:teuthology.orchestra.run.smithi078.stdout:[client.0] 2024-04-24T18:07:00.957 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAoSilmqYUOBBAAXVI0rrfOLOUnxTb+zz9b2w== 2024-04-24T18:07:00.957 INFO:teuthology.orchestra.run.smithi078.stdout:[client.1] 2024-04-24T18:07:00.957 INFO:teuthology.orchestra.run.smithi078.stdout: key = AQAoSilmqTc4BxAAI/hXgahrCmH6RLXgGsMpow== 2024-04-24T18:07:00.958 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T18:07:00.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T18:07:01.050 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.052 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.106 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.157 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.159 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.208 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.210 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T18:07:01.262 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-24T18:07:01.311 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.314 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.363 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.466 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.468 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-24T18:07:01.518 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T18:07:01.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T18:07:01.573 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T18:07:01.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-24T18:07:01.628 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-24T18:07:01.628 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-24T18:07:01.656 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T18:07:01.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-24T18:07:01.778 DEBUG:teuthology.orchestra.run.smithi049:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-04-24T18:07:01.845 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T18:07:01.940 DEBUG:teuthology.orchestra.run.smithi049:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-04-24T18:07:01.967 DEBUG:teuthology.orchestra.run.smithi078:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-04-24T18:07:01.995 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-24T18:07:02.090 DEBUG:teuthology.orchestra.run.smithi078:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-04-24T18:07:02.116 DEBUG:teuthology.orchestra.run.smithi049:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:07:02.118 DEBUG:teuthology.orchestra.run.smithi078:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-24T18:07:02.173 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-24T18:07:02.174 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-24T18:07:02.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-24T18:07:02.176 INFO:tasks.ceph.mon.a:Started 2024-04-24T18:07:02.176 INFO:tasks.ceph.mon.c:Restarting daemon 2024-04-24T18:07:02.176 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-04-24T18:07:02.178 INFO:tasks.ceph.mon.c:Started 2024-04-24T18:07:02.178 INFO:tasks.ceph.mon.b:Restarting daemon 2024-04-24T18:07:02.178 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-04-24T18:07:02.216 INFO:tasks.ceph.mon.b:Started 2024-04-24T18:07:02.216 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-24T18:07:02.216 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-04-24T18:07:02.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-04-24T18:07:02.218 INFO:tasks.ceph.mgr.y:Started 2024-04-24T18:07:02.219 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-24T18:07:02.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-24T18:07:02.220 INFO:tasks.ceph.mgr.x:Started 2024-04-24T18:07:02.221 INFO:tasks.ceph:Setting crush tunables to default 2024-04-24T18:07:02.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd crush tunables default 2024-04-24T18:07:07.641 INFO:teuthology.orchestra.run.smithi049.stderr:adjusted tunables profile to default 2024-04-24T18:07:07.652 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-24T18:07:07.652 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:07.652 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-24T18:07:07.680 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:07.680 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-24T18:07:07.748 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:07.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-24T18:07:07.816 DEBUG:teuthology.orchestra.run.smithi049:> set -ex 2024-04-24T18:07:07.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-24T18:07:07.883 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:07.884 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-04-24T18:07:07.910 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:07.910 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-04-24T18:07:07.977 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:07.977 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-04-24T18:07:08.044 DEBUG:teuthology.orchestra.run.smithi078:> set -ex 2024-04-24T18:07:08.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-04-24T18:07:08.110 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new 24da2d2a-953e-4440-9f6d-fb19cbec9ca1 0 2024-04-24T18:07:08.136 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.144+0000 7f4e118a0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T18:07:08.218 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.226+0000 7f4e118a0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T18:07:08.293 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.301+0000 7f4e118a0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T18:07:08.347 INFO:teuthology.orchestra.run.smithi078.stdout:0 2024-04-24T18:07:08.364 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new a47a3673-abbd-4827-966a-d68b17f3a18f 1 2024-04-24T18:07:08.370 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.378+0000 7f4e118a0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T18:07:08.565 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.573+0000 7f4e118a0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T18:07:08.601 INFO:teuthology.orchestra.run.smithi078.stdout:1 2024-04-24T18:07:08.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new de3910ae-cc81-45d4-8639-cf2415330e64 2 2024-04-24T18:07:08.679 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:08.687+0000 7fc41cfb8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-24T18:07:08.718 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.726+0000 7f4e118a0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T18:07:08.761 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:08.768+0000 7fc41cfb8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-24T18:07:08.790 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.798+0000 7f4e118a0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T18:07:08.836 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:08.843+0000 7fc41cfb8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-24T18:07:08.867 INFO:teuthology.orchestra.run.smithi078.stdout:2 2024-04-24T18:07:08.880 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new 20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0 3 2024-04-24T18:07:08.907 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:08.915+0000 7fc41cfb8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-24T18:07:08.929 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:08.937+0000 7f4e118a0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T18:07:09.008 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:09.016+0000 7f4e118a0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T18:07:09.084 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:09.091+0000 7fc41cfb8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-24T18:07:09.091 INFO:teuthology.orchestra.run.smithi078.stdout:3 2024-04-24T18:07:09.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new 47f1fd84-e601-4e48-8058-bfa9296a9f90 4 2024-04-24T18:07:09.190 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:09.198+0000 7f4e118a0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T18:07:09.235 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:09.243+0000 7fc41cfb8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-24T18:07:09.308 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:09.315+0000 7fc41cfb8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-24T18:07:09.318 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:09.326+0000 7f4e118a0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T18:07:09.341 INFO:teuthology.orchestra.run.smithi078.stdout:4 2024-04-24T18:07:09.354 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new 1796d83b-591c-4c9d-b376-e36a2416a17e 5 2024-04-24T18:07:09.447 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:09.454+0000 7fc41cfb8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-24T18:07:09.525 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:09.532+0000 7fc41cfb8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-24T18:07:09.565 INFO:teuthology.orchestra.run.smithi078.stdout:5 2024-04-24T18:07:09.578 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new 5f203bac-3db4-4129-96cc-63dc06ee5ffa 6 2024-04-24T18:07:09.698 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:09.706+0000 7fc41cfb8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-24T18:07:09.703 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:09.711+0000 7f4e118a0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T18:07:09.809 INFO:teuthology.orchestra.run.smithi078.stdout:6 2024-04-24T18:07:09.821 DEBUG:teuthology.orchestra.run.smithi078:> sudo ceph --cluster ceph osd new 51061805-fbe5-4e80-be6b-dfc37cec0221 7 2024-04-24T18:07:09.828 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:09.836+0000 7fc41cfb8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-24T18:07:09.943 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:09.951+0000 7f4e118a0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T18:07:10.032 INFO:teuthology.orchestra.run.smithi078.stdout:7 2024-04-24T18:07:10.046 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-24T18:07:10.046 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-24T18:07:10.048 INFO:tasks.ceph.osd.0:Started 2024-04-24T18:07:10.048 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-24T18:07:10.048 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-24T18:07:10.050 INFO:tasks.ceph.osd.1:Started 2024-04-24T18:07:10.050 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-24T18:07:10.051 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-24T18:07:10.053 INFO:tasks.ceph.osd.2:Started 2024-04-24T18:07:10.053 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-24T18:07:10.053 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-24T18:07:10.055 INFO:tasks.ceph.osd.3:Started 2024-04-24T18:07:10.055 INFO:tasks.ceph.osd.4:Restarting daemon 2024-04-24T18:07:10.055 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-04-24T18:07:10.057 INFO:tasks.ceph.osd.4:Started 2024-04-24T18:07:10.057 INFO:tasks.ceph.osd.5:Restarting daemon 2024-04-24T18:07:10.057 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-04-24T18:07:10.059 INFO:tasks.ceph.osd.5:Started 2024-04-24T18:07:10.060 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T18:07:10.060 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T18:07:10.062 INFO:tasks.ceph.osd.6:Started 2024-04-24T18:07:10.062 INFO:tasks.ceph.osd.7:Restarting daemon 2024-04-24T18:07:10.062 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-04-24T18:07:10.064 INFO:tasks.ceph.osd.7:Started 2024-04-24T18:07:10.064 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:07:10.206 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:10.214+0000 7f4e118a0180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T18:07:10.249 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:10.257+0000 7fc41cfb8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-24T18:07:10.283 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:10.292+0000 7f4e118a0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T18:07:10.301 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:10.301 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":10,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:07:10.039418+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T18:07:10.039418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:07:10.314 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-24T18:07:10.314 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-24T18:07:10.473 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:10.480+0000 7fc41cfb8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-24T18:07:10.657 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:10.665+0000 7fac76023540 -1 Falling back to public interface 2024-04-24T18:07:10.666 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:10.674+0000 7f5a2374d540 -1 Falling back to public interface 2024-04-24T18:07:10.668 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:10.676+0000 7fdcf02be540 -1 Falling back to public interface 2024-04-24T18:07:10.675 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:10.683+0000 7fe2eed3e540 -1 Falling back to public interface 2024-04-24T18:07:10.675 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:10.683+0000 7fdf1fe48540 -1 Falling back to public interface 2024-04-24T18:07:10.676 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:10.683+0000 7fe2e9160700 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-04-24T18:07:10.680 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:10.688+0000 7f3a33b13540 -1 Falling back to public interface 2024-04-24T18:07:10.684 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:10.691+0000 7fdadfa66540 -1 Falling back to public interface 2024-04-24T18:07:10.688 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:10.695+0000 7f4ac4814540 -1 Falling back to public interface 2024-04-24T18:07:10.711 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:10.719+0000 7fc41cfb8180 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-24T18:07:10.791 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:10.799+0000 7fc41cfb8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-24T18:07:10.948 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:10.956+0000 7f4e118a0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T18:07:11.030 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:11.038+0000 7f4e118a0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T18:07:11.120 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T18:07:11.161 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:11.169+0000 7f4e118a0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T18:07:11.312 INFO:teuthology.misc.health.smithi049.stdout: 2024-04-24T18:07:11.312 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":10,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:07:10.039418+0000","last_up_change":"0.000000","last_in_change":"2024-04-24T18:07:10.039418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:07:11.324 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-04-24T18:07:11.491 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:11.499+0000 7fc41cfb8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-24T18:07:11.577 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:11.584+0000 7fc41cfb8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-24T18:07:11.713 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:11.721+0000 7fc41cfb8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-24T18:07:11.714 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:11.722+0000 7f4e118a0180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T18:07:11.871 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:11.879+0000 7f4e118a0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T18:07:12.032 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:12.040+0000 7f4e118a0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T18:07:12.289 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:12.296+0000 7fc41cfb8180 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-24T18:07:12.456 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:12.463+0000 7fc41cfb8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-24T18:07:12.625 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:12.632+0000 7fc41cfb8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-24T18:07:12.815 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:12.823+0000 7f4e118a0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T18:07:13.031 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:13.039+0000 7f4e118a0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T18:07:13.102 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:13.110+0000 7f4e118a0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T18:07:13.269 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:13.278+0000 7f4e118a0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T18:07:13.485 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:13.492+0000 7fc41cfb8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-24T18:07:13.744 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:13.751+0000 7fc41cfb8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-24T18:07:13.818 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:13.825+0000 7fc41cfb8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-24T18:07:13.991 INFO:tasks.ceph.mgr.y.smithi049.stderr:2024-04-24T18:07:13.999+0000 7fc41cfb8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-24T18:07:14.176 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:14.184+0000 7fac76023540 -1 osd.4 0 log_to_monitors true 2024-04-24T18:07:14.194 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:14.202+0000 7f5a2374d540 -1 osd.5 0 log_to_monitors true 2024-04-24T18:07:14.194 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:14.202+0000 7fdcf02be540 -1 osd.7 0 log_to_monitors true 2024-04-24T18:07:14.197 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:14.205+0000 7f3a33b13540 -1 osd.6 0 log_to_monitors true 2024-04-24T18:07:14.199 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:14.207+0000 7f4ac4814540 -1 osd.2 0 log_to_monitors true 2024-04-24T18:07:14.211 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:14.219+0000 7fe2eed3e540 -1 osd.0 0 log_to_monitors true 2024-04-24T18:07:14.212 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:14.220+0000 7fdf1fe48540 -1 osd.1 0 log_to_monitors true 2024-04-24T18:07:14.222 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:14.230+0000 7fdadfa66540 -1 osd.3 0 log_to_monitors true 2024-04-24T18:07:14.294 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:14.302+0000 7f4dd0356700 -1 mgr.server handle_report got status from non-daemon mon.c 2024-04-24T18:07:14.294 INFO:tasks.ceph.mgr.x.smithi078.stderr:2024-04-24T18:07:14.302+0000 7f4dd0356700 -1 mgr.server handle_report got status from non-daemon mon.b 2024-04-24T18:07:16.319 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:16.327+0000 7fdce86dc700 -1 osd.7 0 waiting for initial osdmap 2024-04-24T18:07:16.322 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:16.330+0000 7fac6cc3e700 -1 osd.4 0 waiting for initial osdmap 2024-04-24T18:07:16.323 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:16.331+0000 7fdcdfcd0700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:16.326 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:16.334+0000 7fac65a35700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:16.334 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:16.342+0000 7f3a2a72e700 -1 osd.6 0 waiting for initial osdmap 2024-04-24T18:07:16.340 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:16.348+0000 7f3a23525700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:16.404 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:16.411+0000 7fdf18266700 -1 osd.1 0 waiting for initial osdmap 2024-04-24T18:07:16.407 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:16.412+0000 7fdad6681700 -1 osd.3 0 waiting for initial osdmap 2024-04-24T18:07:16.407 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:16.415+0000 7fdf0f85a700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:16.415 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:16.423+0000 7fdacf478700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:16.415 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:16.423+0000 7f4abb42f700 -1 osd.2 0 waiting for initial osdmap 2024-04-24T18:07:16.418 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:16.426+0000 7f4ab4226700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:16.427 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:16.434+0000 7fe2e5959700 -1 osd.0 0 waiting for initial osdmap 2024-04-24T18:07:16.430 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:16.438+0000 7fe2de750700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:16.623 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:16.631+0000 7f5a1bb6b700 -1 osd.5 0 waiting for initial osdmap 2024-04-24T18:07:16.627 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:16.635+0000 7f5a1315f700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:07:18.131 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-24T18:07:18.469 INFO:teuthology.misc.health.smithi049.stdout: 2024-04-24T18:07:18.469 INFO:teuthology.misc.health.smithi049.stdout:{"epoch":14,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:07:18.374207+0000","last_up_change":"2024-04-24T18:07:17.349468+0000","last_in_change":"2024-04-24T18:07:10.039418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":43544},{"type":"v1","addr":"172.21.15.78:6825","nonce":43544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":43544},{"type":"v1","addr":"172.21.15.78:6827","nonce":43544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":43544},{"type":"v1","addr":"172.21.15.78:6831","nonce":43544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":43544},{"type":"v1","addr":"172.21.15.78:6829","nonce":43544}]},"public_addr":"172.21.15.78:6825/43544","cluster_addr":"172.21.15.78:6827/43544","heartbeat_back_addr":"172.21.15.78:6831/43544","heartbeat_front_addr":"172.21.15.78:6829/43544","state":["exists","up"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:07:18.483 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-04-24T18:07:18.483 INFO:tasks.ceph:Creating RBD pool 2024-04-24T18:07:18.483 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-24T18:07:19.381 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'rbd' created 2024-04-24T18:07:19.393 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-24T18:07:20.389 INFO:teuthology.orchestra.run.smithi049.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-24T18:07:20.405 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-24T18:07:20.406 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-24T18:07:20.406 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-24T18:07:20.406 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-24T18:07:20.407 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-24T18:07:20.819 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:20.832 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":4,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6832","nonce":42881},{"type":"v1","addr":"172.21.15.78:6833","nonce":42881}]},"active_addr":"172.21.15.78:6833/42881","active_change":"2024-04-24T18:07:13.292299+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4107,"name":"y","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":2512206168}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":53985158}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.78:0","nonce":4199873034}]}]} 2024-04-24T18:07:20.838 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-24T18:07:20.838 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T18:07:20.838 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:07:21.180 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:21.180 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":16,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:07:20.384191+0000","last_up_change":"2024-04-24T18:07:17.349468+0000","last_in_change":"2024-04-24T18:07:10.039418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-24T18:07:18.850062+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-24T18:07:19.353026+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":43544},{"type":"v1","addr":"172.21.15.78:6825","nonce":43544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":43544},{"type":"v1","addr":"172.21.15.78:6827","nonce":43544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":43544},{"type":"v1","addr":"172.21.15.78:6831","nonce":43544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":43544},{"type":"v1","addr":"172.21.15.78:6829","nonce":43544}]},"public_addr":"172.21.15.78:6825/43544","cluster_addr":"172.21.15.78:6827/43544","heartbeat_back_addr":"172.21.15.78:6831/43544","heartbeat_front_addr":"172.21.15.78:6829/43544","state":["exists","up"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:07:21.195 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T18:07:21.195 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:07:21.539 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:21.539 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":17,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:07:21.391166+0000","last_up_change":"2024-04-24T18:07:17.349468+0000","last_in_change":"2024-04-24T18:07:10.039418+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-24T18:07:18.850062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-24T18:07:19.353026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":43544},{"type":"v1","addr":"172.21.15.78:6825","nonce":43544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":43544},{"type":"v1","addr":"172.21.15.78:6827","nonce":43544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":43544},{"type":"v1","addr":"172.21.15.78:6831","nonce":43544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":43544},{"type":"v1","addr":"172.21.15.78:6829","nonce":43544}]},"public_addr":"172.21.15.78:6825/43544","cluster_addr":"172.21.15.78:6827/43544","heartbeat_back_addr":"172.21.15.78:6831/43544","heartbeat_front_addr":"172.21.15.78:6829/43544","state":["exists","up"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:07:21.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T18:07:21.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T18:07:21.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T18:07:21.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T18:07:21.556 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T18:07:21.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T18:07:21.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T18:07:21.557 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T18:07:21.833 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:21.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T18:07:21.847 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:21.848 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T18:07:21.857 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:21.858 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:21.858 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T18:07:21.858 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:21.859 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T18:07:21.859 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T18:07:21.874 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:21.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T18:07:21.878 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:21.879 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T18:07:21.882 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:21.883 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T18:07:22.477 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:22.479 INFO:teuthology.orchestra.run.smithi049.stdout:55834574849 2024-04-24T18:07:22.499 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.3 2024-04-24T18:07:22.512 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.1 2024-04-24T18:07:22.514 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:22.515 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:22.531 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-04-24T18:07:22.539 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-04-24T18:07:22.552 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:22.562 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:22.567 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.7 2024-04-24T18:07:22.577 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.0 2024-04-24T18:07:22.583 INFO:teuthology.orchestra.run.smithi049.stdout:55834574849 2024-04-24T18:07:22.584 INFO:teuthology.orchestra.run.smithi049.stdout:55834574850 2024-04-24T18:07:22.599 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2024-04-24T18:07:22.600 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-04-24T18:07:23.500 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T18:07:23.512 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T18:07:23.532 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T18:07:23.540 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T18:07:23.568 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T18:07:23.578 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T18:07:23.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T18:07:23.601 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T18:07:24.126 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-04-24T18:07:24.143 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.3 2024-04-24T18:07:24.143 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.170 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:24.186 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-04-24T18:07:24.191 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.1 2024-04-24T18:07:24.191 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.203 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.5 2024-04-24T18:07:24.203 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.208 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-04-24T18:07:24.214 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-04-24T18:07:24.228 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.4 2024-04-24T18:07:24.228 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.233 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.7 2024-04-24T18:07:24.233 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.235 INFO:teuthology.orchestra.run.smithi049.stdout:55834574851 2024-04-24T18:07:24.246 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-04-24T18:07:24.250 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574851 for osd.2 2024-04-24T18:07:24.250 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.261 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.0 2024-04-24T18:07:24.261 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.282 INFO:teuthology.orchestra.run.smithi049.stdout:55834574852 2024-04-24T18:07:24.296 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574852 for osd.6 2024-04-24T18:07:24.296 DEBUG:teuthology.parallel:result is None 2024-04-24T18:07:24.296 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T18:07:24.296 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:07:24.667 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:24.669 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:07:24.680 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-24T18:07:23.298826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380308,"kb_used_data":1388,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752594412,"statfs":{"total":773094113280,"available":770656677888,"internally_reserved":0,"allocated":1421312,"data_stored":1580978,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.409357+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:07:20.409357+0000","last_peered":"2024-04-24T18:07:20.409357+0000","last_clean":"2024-04-24T18:07:20.409357+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:07:20.409357+0000","last_undegraded":"2024-04-24T18:07:20.409357+0000","last_fullsized":"2024-04-24T18:07:20.409357+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.409370+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:07:20.409370+0000","last_peered":"2024-04-24T18:07:20.409370+0000","last_clean":"2024-04-24T18:07:20.409370+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:07:20.409370+0000","last_undegraded":"2024-04-24T18:07:20.409370+0000","last_fullsized":"2024-04-24T18:07:20.409370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.407035+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:07:20.407035+0000","last_peered":"2024-04-24T18:07:20.407035+0000","last_clean":"2024-04-24T18:07:20.407035+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:07:20.407035+0000","last_undegraded":"2024-04-24T18:07:20.407035+0000","last_fullsized":"2024-04-24T18:07:20.407035+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.404491+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:07:20.404491+0000","last_peered":"2024-04-24T18:07:20.404491+0000","last_clean":"2024-04-24T18:07:20.404491+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:07:20.404491+0000","last_undegraded":"2024-04-24T18:07:20.404491+0000","last_fullsized":"2024-04-24T18:07:20.404491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.408351+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:07:20.408351+0000","last_peered":"2024-04-24T18:07:20.408351+0000","last_clean":"2024-04-24T18:07:20.408351+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:07:20.408351+0000","last_undegraded":"2024-04-24T18:07:20.408351+0000","last_fullsized":"2024-04-24T18:07:20.408351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":159,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-24T18:07:21.560753+0000","last_change":"2024-04-24T18:07:20.405551+0000","last_active":"2024-04-24T18:07:21.560753+0000","last_peered":"2024-04-24T18:07:21.560753+0000","last_clean":"2024-04-24T18:07:21.560753+0000","last_became_active":"2024-04-24T18:07:20.404611+0000","last_became_peered":"2024-04-24T18:07:20.404611+0000","last_unstale":"2024-04-24T18:07:21.560753+0000","last_undegraded":"2024-04-24T18:07:21.560753+0000","last_fullsized":"2024-04-24T18:07:21.560753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:38.046610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.403860+0000","last_change":"2024-04-24T18:07:20.403369+0000","last_active":"2024-04-24T18:07:20.403860+0000","last_peered":"2024-04-24T18:07:20.403860+0000","last_clean":"2024-04-24T18:07:20.403860+0000","last_became_active":"2024-04-24T18:07:20.403004+0000","last_became_peered":"2024-04-24T18:07:20.403004+0000","last_unstale":"2024-04-24T18:07:20.403860+0000","last_undegraded":"2024-04-24T18:07:20.403860+0000","last_fullsized":"2024-04-24T18:07:20.403860+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:31.704823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.405762+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:07:20.405762+0000","last_peered":"2024-04-24T18:07:20.405762+0000","last_clean":"2024-04-24T18:07:20.405762+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:07:20.405762+0000","last_undegraded":"2024-04-24T18:07:20.405762+0000","last_fullsized":"2024-04-24T18:07:20.405762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:21.110129+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:07:21.110129+0000","last_peered":"2024-04-24T18:07:21.110129+0000","last_clean":"2024-04-24T18:07:21.110129+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:07:21.110129+0000","last_undegraded":"2024-04-24T18:07:21.110129+0000","last_fullsized":"2024-04-24T18:07:21.110129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":225280,"data_stored":642688,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":167936,"data_stored":52320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":642688,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9249999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.089}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0119999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0519999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.051}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":167936,"data_stored":52320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":167936,"data_stored":52320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:07:24.681 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:07:25.013 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:25.015 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:07:25.027 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-04-24T18:07:23.298826+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":11,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380308,"kb_used_data":1388,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752594412,"statfs":{"total":773094113280,"available":770656677888,"internally_reserved":0,"allocated":1421312,"data_stored":1580978,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.409357+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:07:20.409357+0000","last_peered":"2024-04-24T18:07:20.409357+0000","last_clean":"2024-04-24T18:07:20.409357+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:07:20.409357+0000","last_undegraded":"2024-04-24T18:07:20.409357+0000","last_fullsized":"2024-04-24T18:07:20.409357+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.409370+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:07:20.409370+0000","last_peered":"2024-04-24T18:07:20.409370+0000","last_clean":"2024-04-24T18:07:20.409370+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:07:20.409370+0000","last_undegraded":"2024-04-24T18:07:20.409370+0000","last_fullsized":"2024-04-24T18:07:20.409370+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.407035+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:07:20.407035+0000","last_peered":"2024-04-24T18:07:20.407035+0000","last_clean":"2024-04-24T18:07:20.407035+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:07:20.407035+0000","last_undegraded":"2024-04-24T18:07:20.407035+0000","last_fullsized":"2024-04-24T18:07:20.407035+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.404491+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:07:20.404491+0000","last_peered":"2024-04-24T18:07:20.404491+0000","last_clean":"2024-04-24T18:07:20.404491+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:07:20.404491+0000","last_undegraded":"2024-04-24T18:07:20.404491+0000","last_fullsized":"2024-04-24T18:07:20.404491+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.408351+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:07:20.408351+0000","last_peered":"2024-04-24T18:07:20.408351+0000","last_clean":"2024-04-24T18:07:20.408351+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:07:20.408351+0000","last_undegraded":"2024-04-24T18:07:20.408351+0000","last_fullsized":"2024-04-24T18:07:20.408351+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":159,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-04-24T18:07:21.560753+0000","last_change":"2024-04-24T18:07:20.405551+0000","last_active":"2024-04-24T18:07:21.560753+0000","last_peered":"2024-04-24T18:07:21.560753+0000","last_clean":"2024-04-24T18:07:21.560753+0000","last_became_active":"2024-04-24T18:07:20.404611+0000","last_became_peered":"2024-04-24T18:07:20.404611+0000","last_unstale":"2024-04-24T18:07:21.560753+0000","last_undegraded":"2024-04-24T18:07:21.560753+0000","last_fullsized":"2024-04-24T18:07:21.560753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:38.046610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.403860+0000","last_change":"2024-04-24T18:07:20.403369+0000","last_active":"2024-04-24T18:07:20.403860+0000","last_peered":"2024-04-24T18:07:20.403860+0000","last_clean":"2024-04-24T18:07:20.403860+0000","last_became_active":"2024-04-24T18:07:20.403004+0000","last_became_peered":"2024-04-24T18:07:20.403004+0000","last_unstale":"2024-04-24T18:07:20.403860+0000","last_undegraded":"2024-04-24T18:07:20.403860+0000","last_fullsized":"2024-04-24T18:07:20.403860+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:31.704823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:20.405762+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:07:20.405762+0000","last_peered":"2024-04-24T18:07:20.405762+0000","last_clean":"2024-04-24T18:07:20.405762+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:07:20.405762+0000","last_undegraded":"2024-04-24T18:07:20.405762+0000","last_fullsized":"2024-04-24T18:07:20.405762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-04-24T18:07:21.110129+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:07:21.110129+0000","last_peered":"2024-04-24T18:07:21.110129+0000","last_clean":"2024-04-24T18:07:21.110129+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:07:21.110129+0000","last_undegraded":"2024-04-24T18:07:21.110129+0000","last_fullsized":"2024-04-24T18:07:21.110129+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297616,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074224,"statfs":{"total":96636764160,"available":96332005376,"internally_reserved":0,"allocated":225280,"data_stored":642688,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65200000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56599999999999995}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297520,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074320,"statfs":{"total":96636764160,"available":96332103680,"internally_reserved":0,"allocated":167936,"data_stored":52320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28199999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.309}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35999999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297580,"kb_used_data":220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074260,"statfs":{"total":96636764160,"available":96332042240,"internally_reserved":0,"allocated":225280,"data_stored":642688,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.9249999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.306}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.089}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3520000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0119999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":5.0519999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":14.051}]}]},{"osd":0,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":167936,"data_stored":52320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56899999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":164,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":167936,"data_stored":52320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29199999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35499999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76200000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574852,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32300000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56100000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.3260000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91100000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88400000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":152,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":155648,"data_stored":46214,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.629}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72899999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53100000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.755}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.84699999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.86899999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:07:25.028 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T18:07:25.028 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-24T18:07:25.028 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-24T18:07:25.028 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-24T18:07:25.398 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:25.398 INFO:teuthology.orchestra.run.smithi049.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-24T18:07:25.412 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-24T18:07:25.412 INFO:teuthology.run_tasks:Running task exec... 2024-04-24T18:07:25.422 INFO:teuthology.task.exec:Executing custom commands... 2024-04-24T18:07:25.422 INFO:teuthology.task.exec:Running commands on role client.0 host ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:07:25.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool create cache 4' 2024-04-24T18:07:26.492 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cache' created 2024-04-24T18:07:26.506 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier add rbd cache' 2024-04-24T18:07:27.495 INFO:teuthology.orchestra.run.smithi049.stderr:pool 'cache' is now (or already was) a tier of 'rbd' 2024-04-24T18:07:27.509 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier cache-mode cache writeback' 2024-04-24T18:07:28.497 INFO:teuthology.orchestra.run.smithi049.stderr:set cache-mode for pool 'cache' to writeback 2024-04-24T18:07:28.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd tier set-overlay rbd cache' 2024-04-24T18:07:29.503 INFO:teuthology.orchestra.run.smithi049.stderr:overlay for 'rbd' is now (or already was) 'cache' 2024-04-24T18:07:29.516 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_type bloom' 2024-04-24T18:07:30.509 INFO:teuthology.orchestra.run.smithi049.stderr:set pool 3 hit_set_type to bloom 2024-04-24T18:07:30.523 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_count 8' 2024-04-24T18:07:31.512 INFO:teuthology.orchestra.run.smithi049.stderr:set pool 3 hit_set_count to 8 2024-04-24T18:07:31.527 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache hit_set_period 60' 2024-04-24T18:07:32.514 INFO:teuthology.orchestra.run.smithi049.stderr:set pool 3 hit_set_period to 60 2024-04-24T18:07:32.529 DEBUG:teuthology.orchestra.run.smithi049:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'sudo ceph osd pool set cache target_max_objects 250' 2024-04-24T18:07:33.522 INFO:teuthology.orchestra.run.smithi049.stderr:set pool 3 target_max_objects to 250 2024-04-24T18:07:33.536 INFO:teuthology.run_tasks:Running task thrashosds... 2024-04-24T18:07:33.545 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-04-24T18:07:33.545 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy'}, 'ceph': {'conf': {'client': {'rbd cache': True, 'rbd cache policy': 'writearound'}, 'global': {'mon client directed command retry': 5, 'mon warn on pool no app': False, 'ms inject socket failures': 5000}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression algorithm': 'lz4', 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request'], 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'default', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuriw4-testing-20240423.151325-quincy', 'sha1': '330067dbce47655d1f762ef14d546d2fde16a6a9'}} 2024-04-24T18:07:33.545 INFO:tasks.thrashosds:config is {'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-04-24T18:07:33.546 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-04-24T18:07:33.903 INFO:tasks.thrashosds:Beginning thrashosds... 2024-04-24T18:07:33.903 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T18:07:33.903 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:07:34.231 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:34.233 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:07:34.245 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-24T18:07:33.527277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380844,"kb_used_data":1988,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593876,"statfs":{"total":773094113280,"available":770656129024,"internally_reserved":0,"allocated":2035712,"data_stored":1892577,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020756"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.528160+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:07:30.528160+0000","last_peered":"2024-04-24T18:07:30.528160+0000","last_clean":"2024-04-24T18:07:30.528160+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:07:30.528160+0000","last_undegraded":"2024-04-24T18:07:30.528160+0000","last_fullsized":"2024-04-24T18:07:30.528160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.528159+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:07:30.528159+0000","last_peered":"2024-04-24T18:07:30.528159+0000","last_clean":"2024-04-24T18:07:30.528159+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:07:30.528159+0000","last_undegraded":"2024-04-24T18:07:30.528159+0000","last_fullsized":"2024-04-24T18:07:30.528159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527930+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:07:30.527930+0000","last_peered":"2024-04-24T18:07:30.527930+0000","last_clean":"2024-04-24T18:07:30.527930+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:07:30.527930+0000","last_undegraded":"2024-04-24T18:07:30.527930+0000","last_fullsized":"2024-04-24T18:07:30.527930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527906+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:07:30.527906+0000","last_peered":"2024-04-24T18:07:30.527906+0000","last_clean":"2024-04-24T18:07:30.527906+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:07:30.527906+0000","last_undegraded":"2024-04-24T18:07:30.527906+0000","last_fullsized":"2024-04-24T18:07:30.527906+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527550+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:07:30.527550+0000","last_peered":"2024-04-24T18:07:30.527550+0000","last_clean":"2024-04-24T18:07:30.527550+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:07:30.527550+0000","last_undegraded":"2024-04-24T18:07:30.527550+0000","last_fullsized":"2024-04-24T18:07:30.527550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:13:26.133189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527955+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:07:30.527955+0000","last_peered":"2024-04-24T18:07:30.527955+0000","last_clean":"2024-04-24T18:07:30.527955+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:07:30.527955+0000","last_undegraded":"2024-04-24T18:07:30.527955+0000","last_fullsized":"2024-04-24T18:07:30.527955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.512664+0000","last_change":"2024-04-24T18:07:27.510599+0000","last_active":"2024-04-24T18:07:30.512664+0000","last_peered":"2024-04-24T18:07:30.512664+0000","last_clean":"2024-04-24T18:07:30.512664+0000","last_became_active":"2024-04-24T18:07:27.510268+0000","last_became_peered":"2024-04-24T18:07:27.510268+0000","last_unstale":"2024-04-24T18:07:30.512664+0000","last_undegraded":"2024-04-24T18:07:30.512664+0000","last_fullsized":"2024-04-24T18:07:30.512664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:43.433886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527759+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:07:30.527759+0000","last_peered":"2024-04-24T18:07:30.527759+0000","last_clean":"2024-04-24T18:07:30.527759+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:07:30.527759+0000","last_undegraded":"2024-04-24T18:07:30.527759+0000","last_fullsized":"2024-04-24T18:07:30.527759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.512646+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:07:30.512646+0000","last_peered":"2024-04-24T18:07:30.512646+0000","last_clean":"2024-04-24T18:07:30.512646+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:07:30.512646+0000","last_undegraded":"2024-04-24T18:07:30.512646+0000","last_fullsized":"2024-04-24T18:07:30.512646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:25.729450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527869+0000","last_change":"2024-04-24T18:07:20.405551+0000","last_active":"2024-04-24T18:07:30.527869+0000","last_peered":"2024-04-24T18:07:30.527869+0000","last_clean":"2024-04-24T18:07:30.527869+0000","last_became_active":"2024-04-24T18:07:20.404611+0000","last_became_peered":"2024-04-24T18:07:20.404611+0000","last_unstale":"2024-04-24T18:07:30.527869+0000","last_undegraded":"2024-04-24T18:07:30.527869+0000","last_fullsized":"2024-04-24T18:07:30.527869+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:38.046610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.512841+0000","last_change":"2024-04-24T18:07:20.403369+0000","last_active":"2024-04-24T18:07:30.512841+0000","last_peered":"2024-04-24T18:07:30.512841+0000","last_clean":"2024-04-24T18:07:30.512841+0000","last_became_active":"2024-04-24T18:07:20.403004+0000","last_became_peered":"2024-04-24T18:07:20.403004+0000","last_unstale":"2024-04-24T18:07:30.512841+0000","last_undegraded":"2024-04-24T18:07:30.512841+0000","last_fullsized":"2024-04-24T18:07:30.512841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:31.704823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527509+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:07:30.527509+0000","last_peered":"2024-04-24T18:07:30.527509+0000","last_clean":"2024-04-24T18:07:30.527509+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:07:30.527509+0000","last_undegraded":"2024-04-24T18:07:30.527509+0000","last_fullsized":"2024-04-24T18:07:30.527509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527779+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:07:30.527779+0000","last_peered":"2024-04-24T18:07:30.527779+0000","last_clean":"2024-04-24T18:07:30.527779+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:07:30.527779+0000","last_undegraded":"2024-04-24T18:07:30.527779+0000","last_fullsized":"2024-04-24T18:07:30.527779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":299008,"data_stored":680117,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":229376,"data_stored":83598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":680117,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:07:34.246 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:07:34.576 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:34.578 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:07:34.589 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-04-24T18:07:33.527277+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":26,"acting":26,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":26,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2380844,"kb_used_data":1988,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":752593876,"statfs":{"total":773094113280,"available":770656129024,"internally_reserved":0,"allocated":2035712,"data_stored":1892577,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.020756"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.528160+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:07:30.528160+0000","last_peered":"2024-04-24T18:07:30.528160+0000","last_clean":"2024-04-24T18:07:30.528160+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:07:30.528160+0000","last_undegraded":"2024-04-24T18:07:30.528160+0000","last_fullsized":"2024-04-24T18:07:30.528160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.528159+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:07:30.528159+0000","last_peered":"2024-04-24T18:07:30.528159+0000","last_clean":"2024-04-24T18:07:30.528159+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:07:30.528159+0000","last_undegraded":"2024-04-24T18:07:30.528159+0000","last_fullsized":"2024-04-24T18:07:30.528159+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527930+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:07:30.527930+0000","last_peered":"2024-04-24T18:07:30.527930+0000","last_clean":"2024-04-24T18:07:30.527930+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:07:30.527930+0000","last_undegraded":"2024-04-24T18:07:30.527930+0000","last_fullsized":"2024-04-24T18:07:30.527930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527906+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:07:30.527906+0000","last_peered":"2024-04-24T18:07:30.527906+0000","last_clean":"2024-04-24T18:07:30.527906+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:07:30.527906+0000","last_undegraded":"2024-04-24T18:07:30.527906+0000","last_fullsized":"2024-04-24T18:07:30.527906+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527550+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:07:30.527550+0000","last_peered":"2024-04-24T18:07:30.527550+0000","last_clean":"2024-04-24T18:07:30.527550+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:07:30.527550+0000","last_undegraded":"2024-04-24T18:07:30.527550+0000","last_fullsized":"2024-04-24T18:07:30.527550+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:13:26.133189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527955+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:07:30.527955+0000","last_peered":"2024-04-24T18:07:30.527955+0000","last_clean":"2024-04-24T18:07:30.527955+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:07:30.527955+0000","last_undegraded":"2024-04-24T18:07:30.527955+0000","last_fullsized":"2024-04-24T18:07:30.527955+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.512664+0000","last_change":"2024-04-24T18:07:27.510599+0000","last_active":"2024-04-24T18:07:30.512664+0000","last_peered":"2024-04-24T18:07:30.512664+0000","last_clean":"2024-04-24T18:07:30.512664+0000","last_became_active":"2024-04-24T18:07:27.510268+0000","last_became_peered":"2024-04-24T18:07:27.510268+0000","last_unstale":"2024-04-24T18:07:30.512664+0000","last_undegraded":"2024-04-24T18:07:30.512664+0000","last_fullsized":"2024-04-24T18:07:30.512664+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:43.433886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527759+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:07:30.527759+0000","last_peered":"2024-04-24T18:07:30.527759+0000","last_clean":"2024-04-24T18:07:30.527759+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:07:30.527759+0000","last_undegraded":"2024-04-24T18:07:30.527759+0000","last_fullsized":"2024-04-24T18:07:30.527759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.512646+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:07:30.512646+0000","last_peered":"2024-04-24T18:07:30.512646+0000","last_clean":"2024-04-24T18:07:30.512646+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:07:30.512646+0000","last_undegraded":"2024-04-24T18:07:30.512646+0000","last_fullsized":"2024-04-24T18:07:30.512646+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:25.729450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":164,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527869+0000","last_change":"2024-04-24T18:07:20.405551+0000","last_active":"2024-04-24T18:07:30.527869+0000","last_peered":"2024-04-24T18:07:30.527869+0000","last_clean":"2024-04-24T18:07:30.527869+0000","last_became_active":"2024-04-24T18:07:20.404611+0000","last_became_peered":"2024-04-24T18:07:20.404611+0000","last_unstale":"2024-04-24T18:07:30.527869+0000","last_undegraded":"2024-04-24T18:07:30.527869+0000","last_fullsized":"2024-04-24T18:07:30.527869+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:38.046610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.512841+0000","last_change":"2024-04-24T18:07:20.403369+0000","last_active":"2024-04-24T18:07:30.512841+0000","last_peered":"2024-04-24T18:07:30.512841+0000","last_clean":"2024-04-24T18:07:30.512841+0000","last_became_active":"2024-04-24T18:07:20.403004+0000","last_became_peered":"2024-04-24T18:07:20.403004+0000","last_unstale":"2024-04-24T18:07:30.512841+0000","last_undegraded":"2024-04-24T18:07:30.512841+0000","last_fullsized":"2024-04-24T18:07:30.512841+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:41:31.704823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527509+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:07:30.527509+0000","last_peered":"2024-04-24T18:07:30.527509+0000","last_clean":"2024-04-24T18:07:30.527509+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:07:30.527509+0000","last_undegraded":"2024-04-24T18:07:30.527509+0000","last_fullsized":"2024-04-24T18:07:30.527509+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2024-04-24T18:07:30.527779+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:07:30.527779+0000","last_peered":"2024-04-24T18:07:30.527779+0000","last_clean":"2024-04-24T18:07:30.527779+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:07:30.527779+0000","last_undegraded":"2024-04-24T18:07:30.527779+0000","last_fullsized":"2024-04-24T18:07:30.527779+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":6},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":1180736,"data_compressed":17328,"data_compressed_allocated":73728,"data_compressed_original":1146880,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297652,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074188,"statfs":{"total":96636764160,"available":96331968512,"internally_reserved":0,"allocated":299008,"data_stored":680117,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66600000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56999999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297584,"kb_used_data":224,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074256,"statfs":{"total":96636764160,"available":96332038144,"internally_reserved":0,"allocated":229376,"data_stored":83598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66900000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61499999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574853,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297648,"kb_used_data":292,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074192,"statfs":{"total":96636764160,"available":96331972608,"internally_reserved":0,"allocated":299008,"data_stored":680117,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59399999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66800000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56299999999999994}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77700000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26900000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47999999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55200000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574853,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29899999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64100000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.504}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67500000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34300000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53800000000000003}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59799999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58499999999999996}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45500000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297592,"kb_used_data":236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94074248,"statfs":{"total":96636764160,"available":96332029952,"internally_reserved":0,"allocated":241664,"data_stored":89749,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59299999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58199999999999996}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:07:34.591 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T18:07:34.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:epoch 25 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:fsid 5dae32d6-9101-4f7b-a6ef-ca5186fee71b 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:created 2024-04-24T18:07:07.515859+0000 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:modified 2024-04-24T18:07:33.519822+0000 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:crush_version 4 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:full_ratio 0.95 2024-04-24T18:07:34.921 INFO:teuthology.orchestra.run.smithi049.stdout:backfillfull_ratio 0.9 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:nearfull_ratio 0.85 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:require_min_compat_client luminous 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:min_compat_client jewel 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:require_osd_release quincy 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:stretch_mode_enabled false 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:pool 1 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 21 lfor 21/21/21 flags hashpspool tiers 3 read_tier 3 write_tier 3 stripe_width 0 application rbd 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:pool 2 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:pool 3 'cache' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 4 pgp_num 4 autoscale_mode off last_change 25 lfor 21/21/21 flags hashpspool,incomplete_clones tier_of 1 cache_mode writeback target_objects 250 hit_set bloom{false_positive_probability: 0.05, target_size: 0, seed: 0} 60s x8 decay_rate 0 search_last_n 0 stripe_width 0 application rbd 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:max_osd 8 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6801/43474,v1:172.21.15.49:6803/43474] [v2:172.21.15.49:6805/43474,v1:172.21.15.49:6807/43474] exists,up 24da2d2a-953e-4440-9f6d-fb19cbec9ca1 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6800/43473,v1:172.21.15.49:6802/43473] [v2:172.21.15.49:6804/43473,v1:172.21.15.49:6806/43473] exists,up a47a3673-abbd-4827-966a-d68b17f3a18f 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6824/43481,v1:172.21.15.49:6825/43481] [v2:172.21.15.49:6826/43481,v1:172.21.15.49:6827/43481] exists,up de3910ae-cc81-45d4-8639-cf2415330e64 2024-04-24T18:07:34.922 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6816/43475,v1:172.21.15.49:6817/43475] [v2:172.21.15.49:6818/43475,v1:172.21.15.49:6819/43475] exists,up 20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0 2024-04-24T18:07:34.923 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6800/43499,v1:172.21.15.78:6801/43499] [v2:172.21.15.78:6802/43499,v1:172.21.15.78:6803/43499] exists,up 47f1fd84-e601-4e48-8058-bfa9296a9f90 2024-04-24T18:07:34.923 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6808/43500,v1:172.21.15.78:6809/43500] [v2:172.21.15.78:6810/43500,v1:172.21.15.78:6811/43500] exists,up 1796d83b-591c-4c9d-b376-e36a2416a17e 2024-04-24T18:07:34.923 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6824/43544,v1:172.21.15.78:6825/43544] [v2:172.21.15.78:6826/43544,v1:172.21.15.78:6827/43544] exists,up 5f203bac-3db4-4129-96cc-63dc06ee5ffa 2024-04-24T18:07:34.923 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6816/43501,v1:172.21.15.78:6817/43501] [v2:172.21.15.78:6818/43501,v1:172.21.15.78:6819/43501] exists,up 51061805-fbe5-4e80-be6b-dfc37cec0221 2024-04-24T18:07:34.935 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6801/43474,v1:172.21.15.49:6803/43474] [v2:172.21.15.49:6805/43474,v1:172.21.15.49:6807/43474] exists,up 24da2d2a-953e-4440-9f6d-fb19cbec9ca1', 'osd.1 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6800/43473,v1:172.21.15.49:6802/43473] [v2:172.21.15.49:6804/43473,v1:172.21.15.49:6806/43473] exists,up a47a3673-abbd-4827-966a-d68b17f3a18f', 'osd.2 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6824/43481,v1:172.21.15.49:6825/43481] [v2:172.21.15.49:6826/43481,v1:172.21.15.49:6827/43481] exists,up de3910ae-cc81-45d4-8639-cf2415330e64', 'osd.3 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.49:6816/43475,v1:172.21.15.49:6817/43475] [v2:172.21.15.49:6818/43475,v1:172.21.15.49:6819/43475] exists,up 20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0', 'osd.4 up in weight 1 up_from 13 up_thru 18 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6800/43499,v1:172.21.15.78:6801/43499] [v2:172.21.15.78:6802/43499,v1:172.21.15.78:6803/43499] exists,up 47f1fd84-e601-4e48-8058-bfa9296a9f90', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6808/43500,v1:172.21.15.78:6809/43500] [v2:172.21.15.78:6810/43500,v1:172.21.15.78:6811/43500] exists,up 1796d83b-591c-4c9d-b376-e36a2416a17e', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6824/43544,v1:172.21.15.78:6825/43544] [v2:172.21.15.78:6826/43544,v1:172.21.15.78:6827/43544] exists,up 5f203bac-3db4-4129-96cc-63dc06ee5ffa', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) [v2:172.21.15.78:6816/43501,v1:172.21.15.78:6817/43501] [v2:172.21.15.78:6818/43501,v1:172.21.15.78:6819/43501] exists,up 51061805-fbe5-4e80-be6b-dfc37cec0221'] 2024-04-24T18:07:34.936 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-04-24T18:07:35.057 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:35.057 INFO:teuthology.orchestra.run.smithi049.stdout: "name": "mon.a", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster": "ceph", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "admin_socket_mode": "", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_client_required": "cephx, none", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_cluster_required": "cephx", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_debug": "true", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_service_required": "cephx", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_service_ticket_ttl": "240.000000", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "auth_supported": "cephx", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio": "true", 2024-04-24T18:07:35.058 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio_max_queue_depth": "1024", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio_poll_ms": "250", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_aio_reap_max": "16", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_async_discard": "false", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_block_size": "4096", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_aio": "false", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_debug_inflight_ios": "false", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_enable_discard": "false", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_flock_retry": "3", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_flock_retry_interval": "0.100000", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_inject_crash": "0", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_inject_crash_flush_delay": "2", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_ioring": "false", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_ioring_hipri": "false", 2024-04-24T18:07:35.059 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_ioring_sqthread_poll": "false", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_read_buffer_alignment": "4096", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bdev_type": "", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_alloc_size": "1048576", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_allocator": "hybrid", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_buffered_io": "true", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_check_for_zeros": "false", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_check_volume_selector_often": "false", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_compact_log_sync": "false", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-04-24T18:07:35.060 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_log_compact_min_size": "16777216", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_log_replay_check_allocations": "true", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_max_log_runway": "4194304", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_max_prefetch": "1048576", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_min_flush_size": "524288", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_min_log_runway": "1048576", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_replay_recovery": "false", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_shared_alloc_size": "65536", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluefs_sync_write": "false", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_allocation_from_file": "true", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_allocator": "hybrid", 2024-04-24T18:07:35.061 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_blobid_prealloc": "10240", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_create": "true", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_db_create": "false", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_db_path": "", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_db_size": "0", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_path": "", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_preallocate_file": "false", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_size": "107374182400", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_wal_create": "false", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_wal_path": "", 2024-04-24T18:07:35.062 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_block_wal_size": "100663296", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs": "true", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs_env_mirror": "false", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_autotune": "true", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_size": "0", 2024-04-24T18:07:35.063 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cache_type": "2q", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_clone_cow": "true", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_algorithm": "snappy", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_max_blob_size": "0", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_min_blob_size": "0", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_mode": "none", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_csum_type": "crc32c", 2024-04-24T18:07:35.064 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_enforce_settings": "default", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_freelist": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_fsck_abort": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_inject_bug21040": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_inject_read_err": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_legacy_omap": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_prefill": "0.000000", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_small_allocations": "0", 2024-04-24T18:07:35.065 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_default_buffered_read": "true", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_default_buffered_write": "false", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_deferred_batch_ops": "0", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fail_eio": "false", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-04-24T18:07:35.066 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mkfs": "true", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mount": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_umount": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_ignore_data_csum": "false", 2024-04-24T18:07:35.067 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_kvbackend": "rocksdb", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_log_op_age": "5.000000", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_alloc_size": "0", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_blob_size": "0", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_defer_interval": "3.000000", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_max_deferred_txc": "32", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_min_alloc_size": "0", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_nid_prealloc": "1024", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_prefer_deferred_size": "0", 2024-04-24T18:07:35.068 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_qfsck_on_mount": "true", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_retry_disk_reads": "3", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_cf": "true", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L P", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_options": "compression=kNoCompression,max_write_buffer_number=4,min_write_buffer_number_to_merge=1,recycle_log_file_num=4,write_buffer_size=268435456,writable_file_max_buffer_size=0,compaction_readahead_size=2097152,max_background_compactions=2,max_total_wal_size=1073741824", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_rocksdb_options_annex": "", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_coremask": "0x1", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_io_sleep": "5", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_max_io_completion": "0", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_spdk_mem": "512", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_sync_submit_transaction": "false", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_bytes": "67108864", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_cost_per_io": "0", 2024-04-24T18:07:35.069 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_tracing": "false", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_volume_selection_reserved": "0", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "bluestore_zero_block_detection": "false", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_action_update_interval": "2", 2024-04-24T18:07:35.070 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_mount_timeout": "10", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_cluster_require_signatures": "false", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_cluster_require_version": "2", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_require_signatures": "false", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_require_version": "2", 2024-04-24T18:07:35.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_service_require_signatures": "false", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_service_require_version": "2", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "cephx_sign_messages": "true", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "chdir": "", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_acl_type": "", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_asio_thread_count": "2", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_cache_mid": "0.750000", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_cache_size": "16384", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_caps_release_delay": "5", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_check_pool_perm": "true", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_collect_and_send_global_metrics": "false", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_debug_force_sync_read": "false", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_debug_getattr_caps": "false", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_debug_inject_tick_delay": "0", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_die_on_failed_remount": "false", 2024-04-24T18:07:35.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_dirsize_rbytes": "true", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_force_lazyio": "false", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_fs": "", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_inject_fixed_oldest_tid": "false", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_inject_release_failure": "false", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_max_inline_size": "4096", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_max_retries_on_remount_failure": "5", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mds_namespace": "", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_metadata": "", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mount_gid": "-1", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mount_timeout": "300", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mount_uid": "-1", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_mountpoint": "/", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_notify_timeout": "10", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc": "true", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_max_dirty": "104857600", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_max_dirty_age": "5.000000", 2024-04-24T18:07:35.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_max_objects": "1000", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_size": "209715200", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_oc_target_dirty": "8388608", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_permissions": "true", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_quota": "true", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_quota_df": "true", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_readahead_max_bytes": "0", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_readahead_max_periods": "4", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_readahead_min": "131072", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_reconnect_stale": "false", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_shutdown_timeout": "30", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_snapdir": ".snap", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_tick_interval": "1", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_trace": "", 2024-04-24T18:07:35.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_try_dentry_invalidate": "false", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "client_use_faked_inos": "false", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "client_use_random_mds": "false", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_graylog": "false", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_graylog_port": "12201", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_monitors": "default=true", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_syslog": "false", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "clog_to_syslog_level": "info", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "-", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_network": "", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_network_interface": "", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zlib_isal": "false", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zlib_level": "5", 2024-04-24T18:07:35.075 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zlib_winsize": "-15", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "compressor_zstd_level": "1", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_alien_op_num_threads": "6", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_alien_thread_cpu_cores": "", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_osd_obc_lru_size": "10", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_location": "", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_location_hook": "", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_location_hook_timeout": "10", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "daemonize": "false", 2024-04-24T18:07:35.076 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_alienstore": "0/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_allow_any_pool_priority": "false", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_asok": "1/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_asok_assert_abort": "false", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_asserts_on_shutdown": "true", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_auth": "20/20", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_bdev": "1/3", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_bluefs": "1/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_bluestore": "1/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_buffer": "0/1", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_ceph_exporter": "1/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_cephfs_mirror": "0/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_cephsqlite": "0/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_civetweb": "1/10", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_client": "0/5", 2024-04-24T18:07:35.077 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_compressor": "1/5", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_context": "0/1", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_crush": "1/1", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_crypto": "1/5", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_deliberately_leak_memory": "false", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_disable_randomized_ping": "false", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_dpdk": "1/5", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_eventtrace": "1/5", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_filer": "0/1", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_filestore": "1/3", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_finisher": "1/1", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_fuse": "1/5", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_heartbeat_testing_span": "0", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_heartbeatmap": "1/5", 2024-04-24T18:07:35.078 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_immutable_obj_cache": "0/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_javaclient": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_journal": "1/3", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_journaler": "0/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_kstore": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_leveldb": "4/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_lockdep": "0/1", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mclock": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_balancer": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_locker": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_log": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_log_expire": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mds_migrator": "1/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_memdb": "4/5", 2024-04-24T18:07:35.079 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mgr": "2/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mgrc": "1/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_mon": "20/20", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_monc": "0/10", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_ms": "1/1", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_none": "0/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_objclass": "0/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_objectcacher": "0/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_objecter": "0/1", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_optracker": "0/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_osd": "1/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_paxos": "20/20", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_perfcounter": "1/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_prioritycache": "1/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rados": "0/5", 2024-04-24T18:07:35.080 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd_mirror": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd_pwl": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rbd_replay": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_refs": "0/0", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_reserver": "1/1", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw": "1/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_datacache": "1/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rgw_sync": "1/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_rocksdb": "4/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_cache": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_cleaner": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_device": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_journal": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_lba": "0/5", 2024-04-24T18:07:35.081 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_odata": "0/5", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_omap": "0/5", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_onode": "0/5", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_seastore_tm": "0/5", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_striper": "0/1", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_test": "0/5", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_throttle": "1/1", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_timer": "0/1", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "debug_tp": "0/5", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "device_failure_prediction_mode": "none", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_graylog": "false", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_journald": "false", 2024-04-24T18:07:35.082 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_stderr": "true", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "err_to_syslog": "false", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event_tracing": "false", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_addr": "0.0.0.0", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_http_port": "9926", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_prio_limit": "5", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_sort_metrics": "true", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "exporter_stats_period": "5", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "fake_statfs_for_testing": "0", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "fatal_signal_handlers": "true", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "filer_max_purge_ops": "10", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "filer_max_truncate_ops": "128", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_apply_finisher_threads": "1", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_blackhole": "false", 2024-04-24T18:07:35.083 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_btrfs_clone_range": "true", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_btrfs_snap": "true", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_caller_concurrency": "10", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_collect_device_partition_information": "true", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_commit_timeout": "600.000000", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_inject_read_err": "false", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_omap_check": "false", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_random_read_err": "0.000000", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_debug_verify_split": "false", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_dump_file": "", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fadvise": "true", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fail_eio": "true", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fd_cache_shards": "16", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fd_cache_size": "128", 2024-04-24T18:07:35.084 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fiemap": "false", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fiemap_threshold": "4096", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_index_retry_probability": "0.000000", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_inject_stall": "0", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_journal_parallel": "false", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_journal_trailing": "false", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_journal_writeahead": "false", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_kill_at": "0", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size": "0", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs": "0", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-04-24T18:07:35.085 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs_other": "2", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_sync_interval": "5.000000", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size": "0", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_merge_threshold": "-10", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_min_sync_interval": "0.010000", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_odsync_write": "false", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_omap_backend": "rocksdb", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_omap_backend_path": "", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_omap_header_cache_size": "1024", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_ondisk_finisher_threads": "1", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-04-24T18:07:35.086 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_op_thread_timeout": "60", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_op_threads": "2", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_punch_hole": "false", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_bytes": "104857600", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_queue_max_ops": "50", 2024-04-24T18:07:35.087 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_seek_data_hole": "false", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_sloppy_crc": "false", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_splice": "false", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_split_multiple": "2", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_split_rand_factor": "20", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_update_to": "1000", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_enable": "true", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-04-24T18:07:35.088 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_xfs_extsize": "false", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "filestore_zfs_snap": "false", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "fio_dir": "/tmp/fio", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "fsid": "5dae32d6-9101-4f7b-a6ef-ca5186fee71b", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_allow_other": "true", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_atomic_o_trunc": "true", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_big_writes": "true", 2024-04-24T18:07:35.089 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_debug": "false", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_default_permissions": "false", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_disable_pagecache": "false", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_max_write": "0", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_multithreaded": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_require_active_mds": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_set_user_groups": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_splice_move": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_splice_read": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_splice_write": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_syncfs_on_mksnap": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "fuse_use_invalidate_cb": "true", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "gss_target_name": "ceph", 2024-04-24T18:07:35.090 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_file": "", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_inject_failure": "0", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_interval": "5", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "host": "smithi049", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_max_size": "1073741824", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-04-24T18:07:35.091 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "immutable_object_cache_watermark": "0.900000", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "inject_early_sigterm": "false", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "jaeger_tracing_enable": "false", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_aio": "true", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_align_min_size": "65536", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_block_align": "true", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_block_size": "4096", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_dio": "true", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_discard": "false", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_force_aio": "false", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_ignore_corruption": "false", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_max_write_bytes": "10485760", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_max_write_entries": "100", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_replay_from": "0", 2024-04-24T18:07:35.092 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_high_multiple": "0.000000", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_throttle_max_multiple": "0.000000", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_write_header_frequency": "0", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journal_zero_on_create": "false", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journaler_prefetch_periods": "10", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journaler_prezero_periods": "5", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "journaler_write_head_interval": "15", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "key": "", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "keyfile": "", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_backend": "rocksdb", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_default_stripe_size": "65536", 2024-04-24T18:07:35.093 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_fsck_on_mount": "false", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_fsck_on_mount_deep": "true", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_max_bytes": "67108864", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_max_ops": "512", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_nid_prealloc": "1024", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_onode_map_size": "1024", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_sync_submit_transaction": "false", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "kstore_sync_transaction": "false", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_block_size": "65536", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_bloom_size": "0", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_cache_size": "536870912", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_compact_on_mount": "false", 2024-04-24T18:07:35.094 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_compression": "false", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_log": "", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_log_to_ceph_log": "true", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_max_open_files": "0", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_paranoid": "false", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "leveldb_write_buffer_size": "33554432", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "librados_thread_count": "2", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "lockdep": "false", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "lockdep_force_backtrace": "false", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_coarse_timestamps": "true", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_flush_on_exit": "false", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_graylog_host": "127.0.0.1", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_graylog_port": "12201", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_max_new": "1000", 2024-04-24T18:07:35.095 INFO:teuthology.orchestra.run.smithi049.stdout: "log_max_recent": "10000", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_stderr_prefix": "", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_stop_at_utilization": "0.970000", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_file": "true", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_graylog": "false", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_journald": "false", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_stderr": "false", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "log_to_syslog": "false", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "max_rotating_auth_attempts": "10", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_action_on_write_error": "1", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_alternate_name_max": "8192", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_asio_thread_count": "2", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_export_pin": "true", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_dirs": "true", 2024-04-24T18:07:35.096 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_interval": "5", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_fragment_size_max": "100000", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_idle_threshold": "0.000000", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_interval": "10", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_max": "-1", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_max_until": "-1", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_merge_size": "50", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_midchunk": "0.300000", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_min_rebalance": "0.100000", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_min_start": "0.200000", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_minchunk": "0.001000", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_mode": "0", 2024-04-24T18:07:35.097 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_need_max": "1.200000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_need_min": "0.800000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_sample_interval": "3.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_bits": "3", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_rd": "25000.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_size": "10000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_split_wr": "10000.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_target_decay": "10.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_beacon_grace": "15.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_beacon_interval": "4.000000", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_beacon_mon_down_grace": "60", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_memory_limit": "4294967296", 2024-04-24T18:07:35.098 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_mid": "0.700000", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_release_free_interval": "10", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_reservation": "0.050000", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_trim_interval": "1", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cache_trim_threshold": "262144", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_client_delegate_inos_pct": "50", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_client_prealloc_inos": "1000", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_connect_bootstrapping": "false", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_damage_table_max_entries": "10000", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_auth_pins": "false", 2024-04-24T18:07:35.099 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_frag": "false", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_scatterstat": "false", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_debug_subtrees": "false", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_decay_halflife": "5.000000", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_default_dir_hash": "2", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_defer_session_stale": "true", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_deny_all_reconnect": "false", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dir_keys_per_op": "16384", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dir_max_commit_size": "10", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dir_max_entries": "0", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dirstat_min_interval": "1.000000", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_after_rejoin": "false", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_on_map": "false", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_threshold_file": "0", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_early_reply": "true", 2024-04-24T18:07:35.100 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_enable_op_tracker": "true", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_enforce_unique_name": "true", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_distributed": "true", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_random": "true", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_forward_all_requests_to_auth": "false", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_health_cache_threshold": "1.500000", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_health_summarize_threshold": "10", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_heartbeat_grace": "15.000000", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_heartbeat_reset_grace": "1000", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_health_dummy": "false", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_migrator_session_race": "false", 2024-04-24T18:07:35.101 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_join_fs": "", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_journal_format": "1", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_create_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_export_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_import_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_journal_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_journal_expire_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_journal_replay_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_link_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_mdstable_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_openc_at": "0", 2024-04-24T18:07:35.102 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_rename_at": "0", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_events_per_segment": "1024", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_max_events": "-1", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_max_segments": "128", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_pause": "false", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_segment_size": "0", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_skip_corrupt_events": "false", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_log_warn_factor": "2.000000", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_caps_per_client": "1048576", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_completed_flushes": "100000", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_completed_requests": "100000", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_export_size": "20971520", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_file_recover": "32", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_purge_files": "64", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_purge_ops": "8192", 2024-04-24T18:07:35.103 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_snaps_per_dir": "100", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_max_xattr_pairs_size": "65536", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_metrics_update_interval": "2", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_min_caps_per_client": "100", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_min_caps_working_set": "10000", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_numa_node": "-1", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_complaint_time": "30.000000", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_duration": "600", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_size": "20", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_slow_op_size": "20", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_history_slow_op_threshold": "10", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_op_log_threshold": "5", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_ping_grace": "15", 2024-04-24T18:07:35.104 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_ping_interval": "5", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_max_caps": "30000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_max_decay_threshold": "131072", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_recall_warning_threshold": "262144", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_reconnect_timeout": "45.000000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_replay_interval": "1.000000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_root_ino_gid": "0", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_root_ino_uid": "0", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_blocklist_on_evict": "true", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_blocklist_on_timeout": "true", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-04-24T18:07:35.105 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_session_metadata_threshold": "16777216", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_shutdown_check": "0", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_skip_ino": "0", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_sleep_rank_change": "0.000000", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_snap_max_uid": "4294967294", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_snap_min_uid": "0", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_snap_rstat": "false", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_standby_replay_damaged": "false", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_symlink_recovery": "true", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_task_status_update_interval": "2.000000", 2024-04-24T18:07:35.106 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_thrash_exports": "0", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_thrash_fragments": "0", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_tick_interval": "5.000000", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_valgrind_exit": "false", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_verify_backtrace": "1", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_verify_scatter": "false", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_wipe_ino_prealloc": "false", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mds_wipe_sessions": "false", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mempool_debug": "false", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_debug_omit_block_device_write": "false", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_device_bytes": "1073741824", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_page_set": "false", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "memstore_page_size": "65536", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_client_bytes": "134217728", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_client_messages": "512", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_connect_retry_interval": "1.000000", 2024-04-24T18:07:35.107 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_disabled_modules": "", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_max_pg_num_change": "128", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mds_bytes": "134217728", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mds_messages": "128", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mon_bytes": "134217728", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_mon_messages": "128", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_osd_bytes": "536870912", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_osd_messages": "8192", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_pool": "true", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_service_beacon_grace": "60.000000", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_standby_modules": "true", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_stats_period": "5", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_stats_threshold": "5", 2024-04-24T18:07:35.108 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_tick_period": "2", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_accept_timeout_factor": "2.000000", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_allow_pool_delete": "true", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_allow_pool_size_one": "true", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_auth_validate_all_caps": "true", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_bytes": "104857600", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_directed_command_retry": "5", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval": "3.000000", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_hunt_parallel": "3", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_log_interval": "1.000000", 2024-04-24T18:07:35.109 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_ping_interval": "10.000000", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_client_ping_timeout": "30.000000", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_clock_drift_allowed": "1.000000", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_file_level": "debug", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_file": "true", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_graylog": "false", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_journald": "false", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_stderr": "false", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-04-24T18:07:35.110 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_compact_on_bootstrap": "false", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_compact_on_start": "false", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_compact_on_trim": "true", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_con_tracker_persist_interval": "10", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_con_tracker_score_halflife": "43200", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_config_key_max_entry_size": "65536", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_cpu_threads": "4", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_crush_min_required_version": "hammer", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_daemon_bytes": "419430400", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data_avail_crit": "5", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data_avail_warn": "5", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_data_size_warn": "16106127360", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_block_osdmap_trim": "false", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-04-24T18:07:35.111 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_dump_json": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_dump_transactions": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_extra_checks": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_require_pacific": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_no_require_quincy": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_delta_reset_interval": "10.000000", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_dns_srv_name": "ceph-mon", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_down_mkfs_grace": "120", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_election_default_strategy": "1", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_election_timeout": "5.000000", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_elector_ping_divisor": "2", 2024-04-24T18:07:35.112 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_elector_ping_timeout": "2.000000", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_enable_op_tracker": "true", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_fake_pool_delete": "false", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_force_quorum_join": "false", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_globalid_prealloc": "10000", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_detail_to_clog": "false", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_log_update_period": "5", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_max_detail": "50", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_to_clog": "true", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_to_clog_interval": "600", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_host": "172.21.15.49,[v2:172.21.15.49:3301,v1:172.21.15.49:6790],172.21.15.78", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_host_override": "", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_initial_members": "", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-04-24T18:07:35.113 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_keyvaluedb": "rocksdb", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_lease": "5.000000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_log_full_interval": "50", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_log_max": "10000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_log_max_summary": "50", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_log_entries_per_event": "4096", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_log_epochs": "500", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_mdsmap_epochs": "500", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_mgrmap_epochs": "500", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_osd": "10000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_pg_per_osd": "10000", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_pool_pg_num": "65536", 2024-04-24T18:07:35.114 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mds_force_trim_to": "0", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mds_skip_sanity": "false", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_memory_autotune": "true", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_memory_target": "2147483648", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_beacon_grace": "30", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_digest_period": "5", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_inactive_grace": "60", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_mkfs_grace": "240", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_min_osdmap_epochs": "500", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_complaint_time": "30", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_duration": "600", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_size": "20", 2024-04-24T18:07:35.115 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_slow_op_size": "20", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_history_slow_op_threshold": "10", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_op_log_threshold": "5", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_auto_mark_in": "false", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_auto_mark_new_in": "true", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_cache_size": "500", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_cache_size_min": "134217728", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_crush_smoke_test": "true", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_destroyed_out_interval": "600", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_down_out_interval": "600", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-04-24T18:07:35.116 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_force_trim_to": "0", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_full_ratio": "0.950000", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_laggy_halflife": "3600", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_laggy_max_interval": "300", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_laggy_weight": "0.300000", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_max_creating_pgs": "1024", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_max_initial_pgs": "1024", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_min_down_reporters": "2", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_prime_pg_temp": "true", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-04-24T18:07:35.117 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_report_timeout": "900", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_warn_num_repaired": "10", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osd_warn_op_age": "32.000000", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_interval": "10", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_min": "10000", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_osdmap_full_prune_txsize": "100", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_stuck_threshold": "60", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_min_objects": "10000", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_min_per_osd": "0", 2024-04-24T18:07:35.118 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pool_quota_crit_threshold": "0", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_pool_quota_warn_threshold": "0", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_probe_timeout": "2.000000", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_max_change": "0.050000", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_max_osds": "4", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_interval": "86400", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_max_keys": "100", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_scrub_timeout": "300", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_session_timeout": "300", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_smart_report_timeout": "5", 2024-04-24T18:07:35.119 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stat_smooth_intervals": "6", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_pool_min_size": "2", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_pool_size": "4", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_subscribe_interval": "86400.000000", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_debug": "false", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_max_payload_keys": "2000", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_max_payload_size": "1048576", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_provider_kill_at": "0", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_requester_kill_at": "0", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_sync_timeout": "60.000000", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_target_pg_per_osd": "100", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_tick_interval": "5", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_timecheck_interval": "300.000000", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_older_version_delay": "604800", 2024-04-24T18:07:35.120 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_filestore_osds": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_misplaced": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_older_version": "true", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_pool_no_app": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-04-24T18:07:35.121 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_on_too_few_osds": "false", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "monmap": "", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_op_threads": "3", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_buffer_size": "131072", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_cm": "false", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_device_name": "", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_dscp": "96", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_gid_idx": "0", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_local_gid": "", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_polling_us": "1000", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_port_num": "1", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-04-24T18:07:35.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_roce_ver": "1", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_send_buffers": "1024", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_sl": "3", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_support_srq": "true", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_rdma_type": "ib", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_async_reap_threshold": "5", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_before_connect": "false", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_ipv4": "true", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_ipv6": "false", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_msgr1": "true", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_msgr2": "true", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_port_max": "7568", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_port_min": "6800", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_prefer_ipv4": "false", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_retry_count": "3", 2024-04-24T18:07:35.123 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_bind_retry_delay": "5", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_client": "false", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_mds": "false", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_mgr": "false", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_mon": "false", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_blackhole_osd": "false", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_client_mode": "crc secure", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_cluster_mode": "crc secure", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_cluster_type": "", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_compress_secure": "false", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_connection_idle_timeout": "900", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_connection_ready_timeout": "10", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_crc_data": "true", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_crc_header": "true", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_bad_msg": "false", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_bug": "true", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_old_message": "true", 2024-04-24T18:07:35.124 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_skipped_message": "false", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_die_on_unhandled_msg": "false", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_coremask": "0xF", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_devs_allowlist": "", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_enable_tso": "true", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_hugepages": "", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_hw_flow_control": "true", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_lro": "true", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_memory_channel": "4", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_pmd": "", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_port_id": "0", 2024-04-24T18:07:35.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dump_corrupt_message_level": "1", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_dump_on_send": "false", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_initial_backoff": "0.200000", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_delay_max": "1.000000", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_delay_probability": "0.000000", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_delay_type": "", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_internal_delays": "0.000000", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_inject_socket_failures": "5000", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_learn_addr_from_peer": "true", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_max_accept_failures": "4", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_max_backoff": "15.000000", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_mon_client_mode": "secure crc", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_mon_cluster_mode": "secure crc", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_mon_service_mode": "secure crc", 2024-04-24T18:07:35.126 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_osd_compress_min_size": "1024", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_osd_compress_mode": "none", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_osd_compression_algorithm": "snappy zlib zstd lz4", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_pq_min_cost": "65536", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_public_type": "", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_service_mode": "crc secure", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_listen_backlog": "512", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_nodelay": "true", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_tcp_rcvbuf": "0", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "ms_type": "async+posix", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "no_config_file": "false", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_completion_locks_per_session": "32", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_debug_inject_relock_delay": "false", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_inflight_op_bytes": "104857600", 2024-04-24T18:07:35.127 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_inflight_ops": "1024", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_inject_no_watch_ping": "false", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_tick_interval": "5.000000", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "objecter_timeout": "10.000000", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "objectstore_blackhole": "false", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "openssl_engine_opts": "", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_delay_time": "5.000000", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_hist_halflife": "1000", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_max_low_ops": "2", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_max_ops": "4", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_quantize_effort": "0.100000", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_agent_slop": "0.020000", 2024-04-24T18:07:35.128 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_allow_recovery_below_min_size": "true", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_asio_thread_count": "2", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_async_recovery_min_cost": "100", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_auto_mark_unfound_lost": "false", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backfill_retry_interval": "30.000000", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backfill_scan_max": "512", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backfill_scan_min": "64", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backoff_on_degraded": "false", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backoff_on_peering": "false", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_backoff_on_unfound": "true", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_beacon_report_interval": "300", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_duration": "30", 2024-04-24T18:07:35.129 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_max_block_size": "67108864", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_bench_small_size_max_iops": "100", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_blkin_trace_all": "false", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_check_for_log_corruption": "false", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-04-24T18:07:35.130 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_class_update_on_start": "true", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_message_cap": "256", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_message_size_cap": "524288000", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_op_priority": "63", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_client_watch_timeout": "30", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_command_max_records": "256", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_compact_on_start": "false", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_chooseleaf_type": "0", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_initial_weight": "-1.000000", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_update_on_start": "true", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_crush_update_weight_set": "true", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-04-24T18:07:35.131 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_drop_ping_duration": "0", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_feed_pullee": "-1", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_misdirected_ops": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_no_acting_change": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_no_purge_strays": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_op_order": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_pg_log_writeout": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_pretend_recovery_active": "false", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-04-24T18:07:35.132 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_shutdown": "false", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_cached_snaps": "false", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_missing_on_start": "false", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_snaps": "false", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_keys": "1024", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_stride": "524288", 2024-04-24T18:07:35.133 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_default_data_pool_replay_window": "5", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_default_notify_timeout": "30", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep": "0.000000", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_discard_disconnected_ops": "true", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_enable_op_tracker": "true", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_info": "true", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_shutdown": "true", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-04-24T18:07:35.134 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_fast_shutdown_timeout": "15", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_function_tracing": "false", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_grace": "20", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_interval": "6", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_min_peers": "10", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_min_size": "2000", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_stale": "600", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_hit_set_max_size": "100000", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_hit_set_min_size": "1000", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-04-24T18:07:35.135 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_journal_flush_on_shutdown": "true", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_journal_size": "5120", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_kill_backfill_at": "0", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_loop_before_reset_tphandle": "64", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_cache_size": "50", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_dedup": "true", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_message_max": "40", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_message_max_bytes": "10485760", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_map_share_max_epochs": "40", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_attr_name_len": "100", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_attr_size": "0", 2024-04-24T18:07:35.136 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_backfills": "1", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_markdown_count": "5", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_markdown_period": "600", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_object_name_len": "2048", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_object_namespace_len": "256", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_object_size": "134217728", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_omap_entries_per_request": "1024", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pg_blocked_by": "16", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pg_log_entries": "10000", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_pgls": "1024", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_push_cost": "8388608", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_push_objects": "10", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_scrubs": "3", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-04-24T18:07:35.137 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_trimming_pgs": "2", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_write_op_reply_len": "64", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_max_write_size": "90", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_override_recovery_settings": "false", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_profile": "balanced", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-04-24T18:07:35.138 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mclock_skip_benchmark": "false", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_base": "805306368", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_cache_min": "134217728", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-04-24T18:07:35.139 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_target": "4294967296", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_target_autotune": "false", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_min_pg_log_entries": "250", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_min_recovery_priority": "0", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_heartbeat_interval": "30", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_report_interval": "5", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_report_max_in_flight": "2", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_num_cache_shards": "32", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_num_op_tracker_shard": "32", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_numa_auto_affinity": "true", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_numa_node": "-1", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_numa_prefer_iface": "true", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-04-24T18:07:35.140 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objecter_finishers": "1", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objectstore": "bluestore", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objectstore_fuse": "false", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_objectstore_tracing": "false", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_complaint_time": "30.000000", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_duration": "600", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_size": "20", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_slow_op_size": "20", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_log_threshold": "5", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_shards": "0", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_shards_hdd": "5", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_shards_ssd": "8", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_threads_per_shard": "0", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-04-24T18:07:35.141 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_pq_min_cost": "65536", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_queue": "mclock_scheduler", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_queue_cut_off": "high", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_thread_suicide_timeout": "150", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_op_thread_timeout": "15", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_open_classes_on_start": "true", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_os_flags": "0", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_peering_op_priority": "255", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_delete_cost": "1048576", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_delete_priority": "5", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_log_dups_tracked": "3000", 2024-04-24T18:07:35.142 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_log_trim_max": "10000", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_log_trim_min": "100", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_object_context_cache_count": "64", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pg_stat_report_interval_max": "500", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_crush_rule": "-1", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_ec_fast_read": "false", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_bulk": "false", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-04-24T18:07:35.143 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_nodelete": "false", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_flags": "0", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_min_size": "0", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_pg_num": "32", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_pgp_num": "0", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_size": "2", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_default_type": "replicated", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_pool_use_gmt_hitset": "true", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_push_per_object_cost": "1000", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_read_ec_check_for_errors": "false", 2024-04-24T18:07:35.144 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recover_clone_overlap": "true", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recover_clone_overlap_limit": "10", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_cost": "20971520", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_delay_start": "0.000000", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_active": "0", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_active_hdd": "3", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_active_ssd": "10", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_chunk": "8388608", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_max_single_start": "1", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_op_priority": "3", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_op_warn_multiple": "16", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_priority": "5", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_retry_interval": "30.000000", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep": "0.000000", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-04-24T18:07:35.145 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_repair_during_recovery": "false", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_requested_scrub_priority": "120", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_rollback_to_cluster_snap": "", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_auto_repair": "false", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_begin_hour": "0", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_begin_week_day": "0", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_chunk_max": "25", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_chunk_min": "5", 2024-04-24T18:07:35.146 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_cost": "52428800", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_during_recovery": "false", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_end_hour": "0", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_end_week_day": "0", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_event_cost": "4096", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_invalid_stats": "true", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_load_threshold": "0.500000", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_max_interval": "604800.000000", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_max_preemptions": "5", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_min_interval": "86400.000000", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_priority": "5", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_scrub_sleep": "0.000000", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_shutdown_pgref_assert": "false", 2024-04-24T18:07:35.147 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_skip_data_digest": "false", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_smart_report_timeout": "5", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_cost": "1048576", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_priority": "5", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep": "0.000000", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_target_transaction_size": "30", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-04-24T18:07:35.148 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_default_cache_mode": "writeback", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_tracing": "false", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_use_stale_snap": "false", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "osdc_blkin_trace_all": "false", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_kill_at": "0", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_max_join_drift": "10", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_min": "500", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_min_wait": "0.050000", 2024-04-24T18:07:35.149 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_propose_interval": "1.000000", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_service_trim_max": "500", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_service_trim_max_multiplier": "20", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_service_trim_min": "250", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_stash_full_interval": "25", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_trim_max": "500", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "paxos_trim_min": "250", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "perf": "true", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "-", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrv": "", 2024-04-24T18:07:35.150 INFO:teuthology.orchestra.run.smithi049.stdout: "public_bind_addr": "-", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "public_network": "", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "public_network_interface": "", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "qat_compressor_enabled": "false", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rados_mon_op_timeout": "0", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rados_osd_op_timeout": "0", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rados_tracing": "false", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_atime_update_interval": "60", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_balance_parent_reads": "false", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_balance_snap_reads": "false", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_blkin_trace_all": "false", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_blocklist_expire_seconds": "0", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_blocklist_on_break_lock": "true", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache": "true", 2024-04-24T18:07:35.151 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_block_writes_upfront": "false", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_max_dirty": "25165824", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_max_dirty_object": "0", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_policy": "writearound", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_size": "33554432", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_target_dirty": "16777216", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_clone_copy_on_read": "false", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_compression_hint": "none", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_concurrent_management_ops": "10", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_clone_format": "auto", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_data_pool": "", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_features": "61", 2024-04-24T18:07:35.152 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_format": "2", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_map_options": "", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_order": "22", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_pool": "rbd", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_stripe_count": "0", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_default_stripe_unit": "0", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_disable_zero_copy_writes": "true", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_discard_granularity_bytes": "65536", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_enable_alloc_hint": "true", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_io_scheduler": "simple", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_commit_age": "5.000000", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-04-24T18:07:35.153 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_flush_interval": "0", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_order": "24", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_pool": "", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_journal_splay_width": "4", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_localize_parent_reads": "false", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_localize_snap_reads": "false", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-04-24T18:07:35.154 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_type": "simple", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_autotune": "true", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_base": "805306368", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_memory_target": "4294967296", 2024-04-24T18:07:35.155 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_delete_delay": "0", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_replay_delay": "0", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_move_to_trash_on_remove": "false", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_mtime_update_interval": "60", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_non_blocking_aio": "true", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_op_thread_timeout": "60", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_op_threads": "1", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_parent_cache_enabled": "false", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_persistent_cache_mode": "disabled", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_persistent_cache_path": "/tmp", 2024-04-24T18:07:35.156 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_persistent_cache_size": "1073741824", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_plugins": "", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_bps_burst": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_bps_limit": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_exclude_ops": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_iops_burst": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_iops_limit": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_bps_burst": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_bps_limit": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_iops_burst": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_read_iops_limit": "0", 2024-04-24T18:07:35.157 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_schedule_tick_min": "50", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_bps_burst": "0", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_bps_limit": "0", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_iops_burst": "0", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_qos_write_iops_limit": "0", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_quiesce_notification_attempts": "10", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_read_from_replica_policy": "default", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_readahead_max_bytes": "524288", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_readahead_trigger_requests": "10", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_request_timed_out_seconds": "30", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_skip_partial_discard": "true", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-04-24T18:07:35.158 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_tracing": "false", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_validate_names": "true", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd_validate_pool": "true", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "restapi_base_url": "", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "restapi_log_level": "", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_acl_grants_max_num": "100", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_admin_entry": "admin", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_backend_store": "rados", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_barbican_url": "", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_beast_enable_async": "true", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_index_max_aio": "128", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_bucket_quota_ttl": "600", 2024-04-24T18:07:35.159 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cache_enabled": "true", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cache_expiry_interval": "900", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cache_lru_size": "10000", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_content_length_compat": "false", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_copy_obj_progress": "true", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cors_rules_max_num": "100", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_cross_domain_policy": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_default_encryption_key": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_addr": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_ca_path": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_client_cert": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_client_key": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_password": "", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-04-24T18:07:35.160 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_kmip_username": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_require_ssl": "true", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-04-24T18:07:35.161 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_suppress_logs": "true", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_addr": "", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_auth": "token", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_namespace": "", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_prefix": "", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_token_file": "", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_buffersize": "524288", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_low_speed_limit": "1024", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_low_speed_time": "300", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-04-24T18:07:35.162 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_fadvise": "4", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_changes_size": "1000", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_num_shards": "128", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_log_window": "30", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_data_notify_interval_msec": "200", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_data_log_backing": "fifo", 2024-04-24T18:07:35.163 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_region_info_oid": "default.region", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_defer_to_bucket_acls": "", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_data_res": "500.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-04-24T18:07:35.164 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dns_name": "", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dns_s3website_name": "", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_dynamic_resharding": "true", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_gc_threads": "true", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_lc_threads": "true", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_ops_log": "false", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_quota_threads": "true", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_static_website": "false", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enable_usage_log": "false", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_enforce_swift_acls": "true", 2024-04-24T18:07:35.165 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_exit_timeout_secs": "120", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_expose_bucket": "false", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_extended_http_attrs": "", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_frontends": "beast port=7480", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_concurrent_io": "10", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_deferred": "50", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_objs": "32", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_queue_size": "134213632", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_max_trim_chunk": "16", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_obj_min_wait": "7200", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_processor_max_time": "3600", 2024-04-24T18:07:35.166 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_gc_processor_period": "3600", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_get_obj_window_size": "16777216", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_healthcheck_disabling_path": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ignore_get_invalid_range": "false", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_init_timeout": "300", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_domain": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_password": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_password_path": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_project": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_tenant": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_token": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_token_path": "", 2024-04-24T18:07:35.167 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_admin_user": "", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_api_version": "2", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_domain": "", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_password": "", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_project": "", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_tenant": "", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_barbican_user": "", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_implicit_tenants": "false", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_service_token_enabled": "false", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_token_cache_size": "10000", 2024-04-24T18:07:35.168 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_url": "", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_keystone_verify_ssl": "true", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_debug_interval": "-1", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_lock_max_time": "90", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_objs": "32", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_rules": "1000", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_worker": "3", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_max_wp_worker": "3", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lc_thread_delay": "0", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_dnattr": "uid", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-04-24T18:07:35.169 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_searchfilter": "", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ldap_uri": "", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_http_headers": "", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_nonexistent_bucket": "false", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_log_object_name_utc": "false", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_luarocks_location": "/tmp/luarocks", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_attr_name_len": "0", 2024-04-24T18:07:35.170 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_attr_size": "0", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_attrs_num_in_req": "0", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_chunk_size": "4194304", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_concurrent_requests": "1024", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_dynamic_shards": "1999", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_listing_results": "1000", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_notify_retries": "10", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_objs_per_shard": "100000", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_put_param_size": "1048576", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_put_size": "5368709120", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_max_slo_entries": "1000", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_md_log_max_shards": "64", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_md_notify_interval_msec": "200", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-04-24T18:07:35.171 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_mp_lock_max_time": "600", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_multipart_min_part_size": "5242880", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_fhcache_size": "2017", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_lru_lanes": "5", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_max_gc": "300", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_gc_threads": "false", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_lc_threads": "false", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_quota_threads": "false", 2024-04-24T18:07:35.172 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_run_sync_thread": "false", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_num_async_rados_threads": "32", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_num_control_oids": "8", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_numa_node": "-1", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_obj_stripe_size": "4194304", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_objexp_chunk_size": "100", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_objexp_gc_interval": "600", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_objexp_hints_num_shards": "127", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_op_thread_timeout": "600", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_op_tracing": "false", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_opa_token": "", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_opa_url": "", 2024-04-24T18:07:35.173 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_opa_verify_ssl": "true", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_file_path": "", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_rados": "true", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_ops_log_socket_path": "", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_push_interval": "2.000000", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_push_interval_max": "30.000000", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_period_root_pool": ".rgw.root", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_print_continue": "true", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_print_prohibited_content_length": "false", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-04-24T18:07:35.174 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rados_tracing": "false", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_realm": "", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_realm_id": "", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_region": "", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_region_root_pool": ".rgw.root", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_relaxed_region_enforcement": "false", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_request_uri": "", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_batch_size": "64", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_max_aio": "128", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_num_logs": "16", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_reshard_thread_interval": "600", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_resolve_cname": "false", 2024-04-24T18:07:35.175 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_rest_getusage_op_compat": "false", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_run_sync_thread": "true", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_keystone": "false", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_ldap": "false", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_rados": "true", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_auth_use_sts": "false", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_s3_success_create_obj_status": "0", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_scheduler_type": "throttler", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_script_uri": "", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_service_provider_name": "", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_client_id": "", 2024-04-24T18:07:35.176 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_client_secret": "", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_entry": "sts", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_key": "sts", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_max_session_duration": "43200", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_min_session_duration": "900", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sts_token_introspection_url": "", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_account_in_url": "false", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_auth_entry": "auth", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_auth_url": "", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_custom_header": "", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_enforce_content_length": "false", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_need_stats": "true", 2024-04-24T18:07:35.177 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_tenant_name": "", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_token_expiration": "86400", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_url": "", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_url_prefix": "swift", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_swift_versioning_enabled": "false", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_lease_period": "120", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_interval": "1200", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_obj_etag_verify": "false", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_trace_history_size": "4096", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-04-24T18:07:35.178 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_thread_pool_size": "512", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_comment": "", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_createby": "", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_encoding": "", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_flag": "false", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_origin": "", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_sha_unit": "524288", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_torrent_tracker": "", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_trust_forwarded_https": "false", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_log_tick_interval": "30", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_max_shards": "32", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_usage_max_user_shards": "1", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_use_opa_authz": "false", 2024-04-24T18:07:35.179 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_default_quota_max_size": "-1", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_max_buckets": "1000", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_sync_interval": "86400", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_user_unique_email": "true", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_verify_ssl": "true", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_website_routing_rules_max_num": "50", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zone": "", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zone_id": "", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zonegroup": "", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zonegroup_id": "", 2024-04-24T18:07:35.180 INFO:teuthology.orchestra.run.smithi049.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_block_size": "4096", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_shard_bits": "4", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_size": "536870912", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cache_type": "binned_lru", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_collect_compaction_stats": "false", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_collect_extended_stats": "false", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_collect_memory_stats": "false", 2024-04-24T18:07:35.181 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_index_type": "binary_search", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_log_to_ceph_log": "true", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_metadata_block_size": "4096", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_partition_filters": "false", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_perf": "false", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "rotating_keys_renewal_timeout": "10", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "run_dir": "/var/run/ceph", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_block_create": "true", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_cache_lru_size": "67108864", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_default_max_object_size": "16777216", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_device_size": "107374182400", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_init_rewrite_segments_num_per_device": "6", 2024-04-24T18:07:35.182 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_batch_capacity": "16", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_journal_iodepth_limit": "5", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "seastore_segment_size": "67108864", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "setgroup": "", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "setuser": "", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "setuser_match_path": "", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_misplaced_ratio": "0.050000", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "thp": "false", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "threadpool_default_timeout": "60", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "threadpool_empty_queue_max_wait": "2", 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout: "throttler_perf_counter": "true" 2024-04-24T18:07:35.183 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:35.185 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-04-24T18:07:35.320 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a: {} 2024-04-24T18:07:35.330 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b: {} 2024-04-24T18:07:35.340 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c: {} 2024-04-24T18:07:35.340 INFO:teuthology.orchestra.run.smithi049.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T18:07:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T18:07:35.341 INFO:teuthology.orchestra.run.smithi049.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T18:07:35.352 DEBUG:teuthology.orchestra.run.smithi049:> type ceph-objectstore-tool 2024-04-24T18:07:35.368 INFO:teuthology.orchestra.run.smithi049.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-24T18:07:35.368 DEBUG:teuthology.orchestra.run.smithi078:> type ceph-objectstore-tool 2024-04-24T18:07:35.388 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-24T18:07:35.388 INFO:teuthology.run_tasks:Running task rbd_fsx... 2024-04-24T18:07:35.399 INFO:tasks.rbd_fsx:starting rbd_fsx... 2024-04-24T18:07:35.399 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-04-24T18:07:35.400 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:07:35.400 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:07:35.400 INFO:tasks.thrashosds.thrasher:Removing osd 1, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:07:35.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 1 2024-04-24T18:07:35.400 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-04-24T18:07:35.401 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-04-24T18:07:35.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:07:35.401 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-04-24T18:07:35.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:07:35.401 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-04-24T18:07:35.402 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:07:35.402 DEBUG:teuthology.orchestra.run.smithi049:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph_test_librbd_fsx --cluster ceph --id 0 -d -W -R -p 100 -P /home/ubuntu/cephtest/archive -r 1 -w 1 -t 1 -h 1 -l 250000000 -S 0 -N 6000 pool_client.0 image_client.0 2024-04-24T18:07:35.410 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:35.418+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:35.451 INFO:teuthology.orchestra.run.smithi049.stdout:mapped writes DISABLED 2024-04-24T18:07:35.451 INFO:teuthology.orchestra.run.smithi049.stdout:mapped reads DISABLED 2024-04-24T18:07:35.451 INFO:teuthology.orchestra.run.smithi049.stdout:Seed set to 1803 2024-04-24T18:07:35.503 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:35.510+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:35.574 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:35.574 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:35.574 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:35.574 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:35.591 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:07:35.593 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:35.615 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:35.615 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:35.623+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:35.651 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:35.672 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:35.704 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:35.713+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:35.708 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:35.727 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:35.727 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:35.727 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:35.739 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:07:35.748 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:35.783 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:35.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:35.842 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:35.850+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:35.851 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-04-24T18:07:35.859+0000 7fa70e0ea700 -1 mon.a@0(leader).osd e26 definitely_dead 0 2024-04-24T18:07:35.878 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:35.878 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:35.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:07:35.930 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:35.939+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:36.008 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:36.015+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:36.057 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:36.057 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:36.068 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:07:36.108 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:36.117+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:36.209 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:36.217+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:36.235 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:36.235 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:36.245 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:07:36.310 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:36.318+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:36.413 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:36.419+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:36.414 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:36.414 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:36.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:07:36.513 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:36.519+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:36.537 INFO:teuthology.orchestra.run.smithi049.stderr:marked out osd.1. 2024-04-24T18:07:36.541 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:07:36.565 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:36.565 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:36.576 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:07:36.617 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:36.626+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:36.714 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:36.722+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:36.744 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:36.745 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:36.755 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:07:36.840 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:36.848+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:36.915 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:36.924+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:36.923 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:36.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:07:37.017 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:37.026+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:37.103 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:37.113 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:07:37.155 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:37.163+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:37.218 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:37.226+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:37.290 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:37.290 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:37.301 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:07:37.318 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:37.327+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:37.420 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:37.429+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:37.470 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:37.470 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:37.480 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:07:37.521 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:37.530+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:37.617 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:37.617 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:37.622 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:37.630+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:37.627 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:07:37.723 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:37.731+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:37.791 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:37.791 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:37.800 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:07:37.802 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:07:37.842 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:37.850+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:37.925 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:37.934+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:37.949 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:37.965 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:37.965 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:37.967 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:37.975 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:07:37.985 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:38.004 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:38.022 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:38.026 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:38.035+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:38.040 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:38.059 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:38.078 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:38.079 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:38.080 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:38.101 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:38.101 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:38.101 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:38.101 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:38.111 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:07:38.155 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:38.163+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:38.228 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:38.237+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:38.234 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:38.234 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:38.234 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:38.234 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:38.235 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:38.244 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:07:38.329 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:38.338+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:38.411 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:38.411 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:38.411 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:38.411 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [] 2024-04-24T18:07:38.411 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:38.421 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:07:38.466 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:38.474+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:38.531 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:38.540+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:38.547 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:38.547 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:38.547 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:38.547 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:38.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:07:38.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:07:38.641 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:38.650+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:38.722 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:38.722 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:38.722 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:38.723 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:38.723 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:38.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:07:38.733 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:38.742+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:38.834 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:38.843+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:38.899 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:38.899 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:38.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:38.899 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:07:38.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11 4.1f 4:fa654f18:::rbd_id.image_client.0:head [create,call rbd.set_id in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.297376+0000", 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.609092937, 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001488865, 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297376+0000", 2024-04-24T18:07:38.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297376+0000", 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297374+0000", 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297384+0000", 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2410000000000001e-06 2024-04-24T18:07:38.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297399+0000", 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5628e-05 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297407+0000", 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9230000000000002e-06 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297462+0000", 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4864000000000002e-05 2024-04-24T18:07:38.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297588+0000", 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012553 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.297732+0000", 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00014460699999999999 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298416+0000", 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00068364499999999998 2024-04-24T18:07:38.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298793+0000", 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00037742599999999999 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298828+0000", 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4777000000000003e-05 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298865+0000", 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6888000000000003e-05 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.904 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop_reply(client.4468.0:11 4.1f e28/26 ondisk, result = 0)", 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.298726+0000", 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60774261200000002, 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0001428, 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298726+0000", 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298726+0000", 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298726+0000", 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298731+0000", 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1560000000000003e-06 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298741+0000", 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0251999999999999e-05 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298749+0000", 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1810000000000001e-06 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298784+0000", 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5308999999999997e-05 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298790+0000", 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5969999999999999e-06 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298869+0000", 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9259999999999997e-05 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:12 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.298971+0000", 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60749762200000001, 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001668381, 2024-04-24T18:07:38.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 12 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298971+0000", 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298971+0000", 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298970+0000", 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298976+0000", 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:07:38.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298987+0000", 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1148999999999999e-05 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299002+0000", 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4087000000000001e-05 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299042+0000", 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0893999999999998e-05 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299178+0000", 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013531700000000001 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299400+0000", 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022242299999999999 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300349+0000", 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00094871899999999999 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300562+0000", 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000212914 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300599+0000", 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909000000000002e-05 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300640+0000", 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1075999999999998e-05 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop_reply(client.4468.0:12 4.12 e29/27 ondisk, result = 0)", 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.300485+0000", 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60598390800000002, 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000164671, 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300485+0000", 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300485+0000", 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300484+0000", 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300490+0000", 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0800000000000002e-06 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300503+0000", 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2242000000000001e-05 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300509+0000", 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9440000000000002e-06 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300552+0000", 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3191e-05 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300558+0000", 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.533e-06 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300650+0000", 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.1503999999999997e-05 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:14 4.1f 4:fa654f18:::rbd_id.image_client.0:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.304173+0000", 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60229633199999999, 2024-04-24T18:07:38.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00025350999999999998, 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 14 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304173+0000", 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304173+0000", 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304172+0000", 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304183+0000", 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1007e-05 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304195+0000", 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2703e-05 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304202+0000", 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1760000000000004e-06 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304250+0000", 2024-04-24T18:07:38.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7308000000000002e-05 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304338+0000", 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8276000000000007e-05 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304426+0000", 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8232000000000005e-05 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.921 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:15 4.1f 4:fa654f18:::rbd_id.image_client.0:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.304599+0000", 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60187051599999997, 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000368148, 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 15 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304599+0000", 2024-04-24T18:07:38.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304599+0000", 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304598+0000", 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304609+0000", 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1007e-05 2024-04-24T18:07:38.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304621+0000", 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2805e-05 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304629+0000", 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2200000000000003e-06 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304659+0000", 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0365000000000002e-05 2024-04-24T18:07:38.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304720+0000", 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1309999999999994e-05 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.304967+0000", 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024648200000000002 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:16 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call rbd.get_size in=11b,call,call rbd.get_features in=15b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.925 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.305231+0000", 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60123827100000005, 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000264419, 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 16 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305231+0000", 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305231+0000", 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305230+0000", 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305235+0000", 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5909999999999998e-06 2024-04-24T18:07:38.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305245+0000", 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9839999999999996e-06 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305250+0000", 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8279999999999999e-06 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305294+0000", 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3285999999999999e-05 2024-04-24T18:07:38.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305359+0000", 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4950000000000007e-05 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305495+0000", 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013660500000000001 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:17 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.929 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.305698+0000", 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60077088099999998, 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00025433999999999998, 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 17 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305698+0000", 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305698+0000", 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305698+0000", 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305703+0000", 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0340000000000004e-06 2024-04-24T18:07:38.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305712+0000", 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4290000000000007e-06 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305719+0000", 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9450000000000004e-06 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305763+0000", 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3640999999999998e-05 2024-04-24T18:07:38.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305840+0000", 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7539000000000004e-05 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.305953+0000", 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000112413 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:18 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call,call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.933 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.306105+0000", 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.60036415499999995, 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00026690699999999998, 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 18 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306105+0000", 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306105+0000", 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306104+0000", 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306108+0000", 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8449999999999996e-06 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306116+0000", 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.391e-06 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306119+0000", 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2100000000000002e-06 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306150+0000", 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1075999999999999e-05 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306227+0000", 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7010999999999998e-05 2024-04-24T18:07:38.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306372+0000", 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00014501800000000001 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:19 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.306608+0000", 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59986119100000002, 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00017782900000000001, 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.937 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 19 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306608+0000", 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306608+0000", 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306607+0000", 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306611+0000", 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.143e-06 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306619+0000", 2024-04-24T18:07:38.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5970000000000003e-06 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306622+0000", 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2890000000000002e-06 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306653+0000", 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0701000000000003e-05 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306703+0000", 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9731999999999998e-05 2024-04-24T18:07:38.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.306786+0000", 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2941000000000004e-05 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:20 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call rbd.get_size in=11b,call rbd.get_features in=15b,call rbd.get_flags in=12b,call,call lock.get_info in=12b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.307211+0000", 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59925816099999996, 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00049737299999999996, 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 20 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307211+0000", 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307211+0000", 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307210+0000", 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307216+0000", 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0340000000000002e-06 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307227+0000", 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1430000000000001e-05 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307235+0000", 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5499999999999997e-06 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307269+0000", 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4085999999999999e-05 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307351+0000", 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2453999999999994e-05 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.307708+0000", 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00035690100000000002 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:21 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call,call rbd.parent_overlap_get in=21b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.308056+0000", 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59841275000000005, 2024-04-24T18:07:38.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000395213, 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 21 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308056+0000", 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308056+0000", 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308055+0000", 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308061+0000", 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3940000000000004e-06 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308072+0000", 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1123e-05 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308080+0000", 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7389999999999999e-06 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308114+0000", 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4842999999999998e-05 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308191+0000", 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6186000000000005e-05 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308452+0000", 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00026086799999999998 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:22 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call rbd.metadata_list in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.308782+0000", 2024-04-24T18:07:38.949 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59768729799999998, 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00029628199999999998, 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 22 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308782+0000", 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308782+0000", 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308781+0000", 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308791+0000", 2024-04-24T18:07:38.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0359e-05 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308803+0000", 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1338999999999999e-05 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308811+0000", 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5169999999999999e-06 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308851+0000", 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9987000000000002e-05 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.308934+0000", 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2291000000000002e-05 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309078+0000", 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00014450899999999999 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:24 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.309728+0000", 2024-04-24T18:07:38.953 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59674151399999997, 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024761599999999998, 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 24 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309728+0000", 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309728+0000", 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309727+0000", 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309731+0000", 2024-04-24T18:07:38.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3529999999999997e-06 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309740+0000", 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.1130000000000007e-06 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309745+0000", 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4170000000000002e-06 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309782+0000", 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7212000000000002e-05 2024-04-24T18:07:38.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309859+0000", 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7405999999999997e-05 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.309975+0000", 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000115835 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:25 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [watch watch cookie 139789911209104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:38.957 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.310198+0000", 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59627094400000002, 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011323819999999999, 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 25 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310198+0000", 2024-04-24T18:07:38.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310198+0000", 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310197+0000", 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310202+0000", 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:07:38.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310212+0000", 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0169e-05 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310218+0000", 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2709999999999998e-06 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310250+0000", 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2360999999999997e-05 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310327+0000", 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6958000000000004e-05 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310479+0000", 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00015179599999999999 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.310969+0000", 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00049035599999999995 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:38.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311267+0000", 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000297798 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311293+0000", 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5374000000000001e-05 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311331+0000", 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7874999999999999e-05 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop_reply(client.4468.0:25 4.12 e29/27 ondisk, result = 0)", 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.311223+0000", 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59524560299999996, 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000110197, 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311223+0000", 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:38.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311223+0000", 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311223+0000", 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311227+0000", 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3259999999999997e-06 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311236+0000", 2024-04-24T18:07:38.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8529999999999998e-06 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311240+0000", 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1200000000000004e-06 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311260+0000", 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9545999999999999e-05 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311264+0000", 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4569999999999998e-06 2024-04-24T18:07:38.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.311334+0000", 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9574999999999997e-05 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:38.966 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:38.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:07:38.968 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:38.976+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:39.068 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:39.077+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:39.092 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:39.092 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:39.092 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:39.092 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:39.102 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:07:39.169 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:39.178+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:39.225 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:39.225 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:39.225 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:39.225 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:39.225 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:39.235 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:07:39.278 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:39.286+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:39.284 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:07:39.363 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753943167999999, 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:07:39.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753874115999999, 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:07:39.369 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:07:39.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:07:39.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:07:39.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:07:39.374 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753849701, 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753826552, 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.380 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:07:39.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:07:39.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753745983999998, 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.385 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:39.390 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753721173999999, 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.395 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753701062000001, 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.396 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753630398999999, 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.401 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.402 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.753618626000002, 2024-04-24T18:07:39.406 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.407 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:07:39.408 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:07:39.409 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:07:39.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.411 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.333635264000002, 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0070057069999999999, 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 132 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036533+0000", 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036535+0000", 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:07:39.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036539+0000", 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3459999999999998e-06 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036541+0000", 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9520000000000001e-06 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037943+0000", 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001402401 2024-04-24T18:07:39.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038021+0000", 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8071000000000005e-05 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038150+0000", 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012869199999999999 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.039054+0000", 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00090387400000000004 2024-04-24T18:07:39.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043447+0000", 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043929110000000002 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043497+0000", 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0420999999999997e-05 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043539+0000", 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2255000000000001e-05 2024-04-24T18:07:39.416 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.333333014000001, 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0068998769999999996, 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.417 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:39.418 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036839+0000", 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9739999999999998e-06 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036844+0000", 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3079999999999998e-06 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036846+0000", 2024-04-24T18:07:39.419 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4099999999999998e-06 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040072+0000", 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003225805 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040118+0000", 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6026999999999998e-05 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040191+0000", 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2983000000000002e-05 2024-04-24T18:07:39.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042013+0000", 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001822118 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043702+0000", 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0016890329999999999 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043721+0000", 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8754999999999999e-05 2024-04-24T18:07:39.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043736+0000", 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4725e-05 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.333296796999999, 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.422 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.423 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:07:39.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.426 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.333249615, 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.427 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:39.428 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:07:39.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:07:39.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.431 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.333224494, 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.432 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:39.433 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:07:39.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:07:39.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.333172359999999, 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.437 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:07:39.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:07:39.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:07:39.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.333108895999999, 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.443 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:07:39.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.445 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:07:39.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.821938596999999, 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073643390000000001, 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 156 2024-04-24T18:07:39.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:39.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548239+0000", 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8899999999999996e-06 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548246+0000", 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9370000000000004e-06 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548259+0000", 2024-04-24T18:07:39.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549586+0000", 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001326743 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549641+0000", 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5856999999999997e-05 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549713+0000", 2024-04-24T18:07:39.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1970000000000004e-05 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550899+0000", 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001185567 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555532+0000", 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0046325760000000002 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555573+0000", 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1730000000000002e-05 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555595+0000", 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1324000000000001e-05 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.821903224, 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.453 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:07:39.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:07:39.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.821878197, 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:07:39.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:07:39.463 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.821798959999999, 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:07:39.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:07:39.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:39.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:39.469 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:39.471 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:39.479+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:39.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:07:39.571 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:39.580+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:39.609 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:39.609 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:39.619 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:07:39.672 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:39.681+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:39.748 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:39.748 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:39.748 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:39.758 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:07:39.799 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:39.808+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:39.874 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:39.883+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "MOSDPGPush(2.0 29/28 [PushOp(2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'116, size: 262688, copy_subset: [0~262688], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:262688, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:38.295504+0000", 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.64097257, 2024-04-24T18:07:39.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098017500000000001, 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295504+0000", 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295504+0000", 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295503+0000", 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295787+0000", 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028404199999999998 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295809+0000", 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1637999999999998e-05 2024-04-24T18:07:39.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295828+0000", 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9018999999999999e-05 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295898+0000", 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0375000000000003e-05 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295912+0000", 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3162999999999999e-05 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.296484+0000", 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00057271500000000001 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "MOSDPGPush(2.0 29/28 [PushOp(2:30306672:devicehealth::main.db.0000000000000000:head, version: 17'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(2:30306672:devicehealth::main.db.0000000000000000:head@17'117, size: 327680, copy_subset: [0~327680], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:327680, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:38.298100+0000", 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.6383767310000001, 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00078320799999999997, 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:39.932 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298100+0000", 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298100+0000", 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298099+0000", 2024-04-24T18:07:39.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298400+0000", 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030102999999999998 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298415+0000", 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4348999999999999e-05 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:39.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298428+0000", 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3059e-05 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298474+0000", 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6196999999999998e-05 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298479+0000", 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.775e-06 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:39.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298883+0000", 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00040442400000000002 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:39.936 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:39.939 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:07:39.975 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:39.984+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:40.057 INFO:teuthology.orchestra.run.smithi049.stdout:1 write 0xb45fc20 thru 0xb46e1f3 (0xe5d4 bytes) 2024-04-24T18:07:40.060 INFO:teuthology.orchestra.run.smithi049.stdout:truncating to largest ever: 0xbc12719 2024-04-24T18:07:40.061 INFO:teuthology.orchestra.run.smithi049.stdout:2 trunc from 0xb46e1f4 to 0xbc12719 2024-04-24T18:07:40.071 INFO:teuthology.orchestra.run.smithi049.stdout:3 trunc from 0xbc12719 to 0x74862b5 2024-04-24T18:07:40.076 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:40.085+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:40.081 INFO:teuthology.orchestra.run.smithi049.stdout:5 write 0x25dfbb7 thru 0x25e8407 (0x8851 bytes) 2024-04-24T18:07:40.084 INFO:teuthology.orchestra.run.smithi049.stdout:6 punch from 0x6ba7498 to 0x6ba7d9c, (0x904 bytes) 2024-04-24T18:07:40.084 INFO:teuthology.orchestra.run.smithi049.stdout:7 clone 1 order 23 su 65536 sc 12 2024-04-24T18:07:40.091 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:07:40.125 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:40.125 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:40.125 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:40.136 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:07:40.177 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:40.185+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:40.242 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:40.262 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:40.278 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:40.287+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:40.282 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:40.304 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:40.321 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:40.322 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:40.325 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:40.330 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:07:40.344 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:40.364 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:40.379 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:40.388+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:40.384 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:40.386 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:40.413 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:40.428 INFO:teuthology.orchestra.run.smithi049.stdout:9 write 0xcd76788 thru 0xcd82be2 (0xc45b bytes) 2024-04-24T18:07:40.433 INFO:teuthology.orchestra.run.smithi049.stdout:10 read 0x3a51fa9 thru 0x3a5746f (0x54c7 bytes) 2024-04-24T18:07:40.433 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:40.433 INFO:teuthology.orchestra.run.smithi049.stdout:12 write 0x3de17c8 thru 0x3ded09b (0xb8d4 bytes) 2024-04-24T18:07:40.447 INFO:teuthology.orchestra.run.smithi049.stdout:13 trunc from 0xcd82be3 to 0x445b6f2 2024-04-24T18:07:40.471 INFO:teuthology.orchestra.run.smithi049.stdout:truncating to largest ever: 0xe7d0485 2024-04-24T18:07:40.480 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:40.489+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:40.482 INFO:teuthology.orchestra.run.smithi049.stdout:14 trunc from 0x445b6f2 to 0xe7d0485 2024-04-24T18:07:40.487 INFO:teuthology.orchestra.run.smithi049.stdout:16 punch from 0x62953fb to 0x6299d48, (0x494d bytes) 2024-04-24T18:07:40.487 INFO:teuthology.orchestra.run.smithi049.stdout:17 write 0x13c6abd thru 0x13c7f20 (0x1464 bytes) 2024-04-24T18:07:40.487 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:40.487 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:40.488 INFO:teuthology.orchestra.run.smithi049.stdout:19 read 0xfcab1f thru 0xfd8628 (0xdb0a bytes) 2024-04-24T18:07:40.489 INFO:teuthology.orchestra.run.smithi049.stdout:20 read 0x4ae709b thru 0x4af0771 (0x96d7 bytes) 2024-04-24T18:07:40.489 INFO:teuthology.orchestra.run.smithi049.stdout:21 writesame 0x64d48f8 thru 0x64e034e data_size 0x1(0xba57 bytes) 2024-04-24T18:07:40.500 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:07:40.516 INFO:teuthology.orchestra.run.smithi049.stdout:22 read 0x694ce08 thru 0x695b1af (0xe3a8 bytes) 2024-04-24T18:07:40.516 INFO:teuthology.orchestra.run.smithi049.stdout:23 write 0x9790427 thru 0x9795d20 (0x58fa bytes) 2024-04-24T18:07:40.518 INFO:teuthology.orchestra.run.smithi049.stdout:24 write 0x92992c8 thru 0x92a0df7 (0x7b30 bytes) 2024-04-24T18:07:40.521 INFO:teuthology.orchestra.run.smithi049.stdout:25 write 0x9cae667 thru 0x9cb6a95 (0x842f bytes) 2024-04-24T18:07:40.541 INFO:teuthology.orchestra.run.smithi049.stdout:26 trunc from 0xe7d0485 to 0xb4038 2024-04-24T18:07:40.571 INFO:teuthology.orchestra.run.smithi049.stdout:28 writesame 0xa554f1c thru 0xa559e4c data_size 0x13(0x4f31 bytes) 2024-04-24T18:07:40.571 INFO:teuthology.orchestra.run.smithi049.stdout:29 write 0x29ab999 thru 0x29b17a5 (0x5e0d bytes) 2024-04-24T18:07:40.573 INFO:teuthology.orchestra.run.smithi049.stdout:30 write 0x9439638 thru 0x943e50c (0x4ed5 bytes) 2024-04-24T18:07:40.581 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:40.589+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:40.582 INFO:teuthology.orchestra.run.smithi049.stdout:31 write 0xbc8ac50 thru 0xbc99b64 (0xef15 bytes) 2024-04-24T18:07:40.594 INFO:teuthology.orchestra.run.smithi049.stdout:32 trunc from 0xbc99b65 to 0x3c40781 2024-04-24T18:07:40.610 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:40.611 INFO:teuthology.orchestra.run.smithi049.stdout:34 write 0x2523e7b thru 0x2533664 (0xf7ea bytes) 2024-04-24T18:07:40.612 INFO:teuthology.orchestra.run.smithi049.stdout:35 read 0x367a548 thru 0x36834ad (0x8f66 bytes) 2024-04-24T18:07:40.626 INFO:teuthology.orchestra.run.smithi049.stdout:37 writesame 0x9f67fe9 thru 0x9f6f4cf data_size 0x1(0x74e7 bytes) 2024-04-24T18:07:40.626 INFO:teuthology.orchestra.run.smithi049.stdout:38 read 0x8a860c3 thru 0x8a931d4 (0xd112 bytes) 2024-04-24T18:07:40.626 INFO:teuthology.orchestra.run.smithi049.stdout:40 punch from 0x67e6f07 to 0x67f4338, (0xd431 bytes) 2024-04-24T18:07:40.626 INFO:teuthology.orchestra.run.smithi049.stdout:42 write 0x2d4db0b thru 0x2d4e11e (0x614 bytes) 2024-04-24T18:07:40.627 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:40.627 INFO:teuthology.orchestra.run.smithi049.stdout:44 clone 2 order 20 su 131072 sc 15 2024-04-24T18:07:40.661 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:40.662 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:40.672 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:07:40.714 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:40.722+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:40.782 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:40.790+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:40.839 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:40.840 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:40.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:07:40.894 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:40.903+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:40.978 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:40.978 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:40.983 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:40.992+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:40.988 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:07:41.084 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:41.092+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:41.157 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:41.157 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:41.167 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:07:41.185 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:41.193+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:41.286 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:41.294+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:41.296 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:07:41.336 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:41.336 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:41.349 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:07:41.389 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:41.398+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:41.488 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:41.496+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:41.520 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:41.520 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:41.530 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:07:41.558 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:07:41.558 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:07:41.558 INFO:tasks.thrashosds.thrasher:inject_pause on osd.0 2024-04-24T18:07:41.558 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-24T18:07:41.559 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-24T18:07:41.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok config set filestore_inject_stall 3 2024-04-24T18:07:41.588 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:41.597+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:41.607 INFO:teuthology.orchestra.run.smithi049.stdout:flattening image image_client.0-clone1 2024-04-24T18:07:41.674 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:41.674 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:41.684 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:07:41.710 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-24T18:07:41.710 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:41.721 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #0, image image_client.0 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent1 2024-04-24T18:07:41.725 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:41.733+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:41.790 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:41.799+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:41.847 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:41.847 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:41.851 INFO:teuthology.orchestra.run.smithi049.stdout:45 write 0xe42706 thru 0xe4bb6d (0x9468 bytes) 2024-04-24T18:07:41.858 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:07:41.860 INFO:teuthology.orchestra.run.smithi049.stdout:46 punch from 0x3b5de95 to 0x3b6591e, (0x7a89 bytes) 2024-04-24T18:07:41.860 INFO:teuthology.orchestra.run.smithi049.stdout:47 read 0x8cec37f thru 0x8cf9dfd (0xda7f bytes) 2024-04-24T18:07:41.860 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:41.865 INFO:teuthology.orchestra.run.smithi049.stdout:50 trunc from 0x9f6f4d0 to 0xe51fbf9 2024-04-24T18:07:41.870 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:41.870 INFO:teuthology.orchestra.run.smithi049.stdout:54 punch from 0x39cf4b7 to 0x39d08f5, (0x143e bytes) 2024-04-24T18:07:41.870 INFO:teuthology.orchestra.run.smithi049.stdout:55 write 0xb218371 thru 0xb21d815 (0x54a5 bytes) 2024-04-24T18:07:41.874 INFO:teuthology.orchestra.run.smithi049.stdout:56 punch from 0x369f7c3 to 0x36a0f41, (0x177e bytes) 2024-04-24T18:07:41.875 INFO:teuthology.orchestra.run.smithi049.stdout:57 read 0x4f41a66 thru 0x4f41db0 (0x34b bytes) 2024-04-24T18:07:41.875 INFO:teuthology.orchestra.run.smithi049.stdout:58 punch from 0x32bb892 to 0x32c04a9, (0x4c17 bytes) 2024-04-24T18:07:41.875 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:41.875 INFO:teuthology.orchestra.run.smithi049.stdout:60 read 0xdc8cfb4 thru 0xdc8ed60 (0x1dad bytes) 2024-04-24T18:07:41.875 INFO:teuthology.orchestra.run.smithi049.stdout:62 writesame 0xd157172 thru 0xd15de35 data_size 0x2(0x6cc4 bytes) 2024-04-24T18:07:41.883 INFO:teuthology.orchestra.run.smithi049.stdout:63 write 0xcfd0182 thru 0xcfd382f (0x36ae bytes) 2024-04-24T18:07:41.887 INFO:teuthology.orchestra.run.smithi049.stdout:64 read 0xb02654f thru 0xb035d61 (0xf813 bytes) 2024-04-24T18:07:41.887 INFO:teuthology.orchestra.run.smithi049.stdout:65 write 0xced7f3e thru 0xcee5d98 (0xde5b bytes) 2024-04-24T18:07:41.891 INFO:teuthology.orchestra.run.smithi049.stdout:66 read 0x4d69cf2 thru 0x4d7129e (0x75ad bytes) 2024-04-24T18:07:41.891 INFO:teuthology.orchestra.run.smithi049.stdout:68 write 0xc7fe9dd thru 0xc80b56f (0xcb93 bytes) 2024-04-24T18:07:41.897 INFO:teuthology.orchestra.run.smithi049.stdout:69 writesame 0xe1557d5 thru 0xe15851e data_size 0x22(0x2d4a bytes) 2024-04-24T18:07:41.899 INFO:teuthology.orchestra.run.smithi049.stdout:73 writesame 0x7b10d2d thru 0x7b15b8f data_size 0x1(0x4e63 bytes) 2024-04-24T18:07:41.900 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:41.909+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:41.910 INFO:teuthology.orchestra.run.smithi049.stdout:76 write 0x2701bca thru 0x2707da7 (0x61de bytes) 2024-04-24T18:07:41.916 INFO:teuthology.orchestra.run.smithi049.stdout:77 writesame 0x42c6a77 thru 0x42d24d3 data_size 0x5d(0xba5d bytes) 2024-04-24T18:07:41.920 INFO:teuthology.orchestra.run.smithi049.stdout:78 writesame 0x4ab8eed thru 0x4ac15f2 data_size 0x15(0x8706 bytes) 2024-04-24T18:07:41.926 INFO:teuthology.orchestra.run.smithi049.stdout:79 punch from 0xcbbcc4c to 0xcbcc25e, (0xf612 bytes) 2024-04-24T18:07:41.926 INFO:teuthology.orchestra.run.smithi049.stdout:80 writesame 0x2d3efdd thru 0x2d4aab2 data_size 0x5(0xbad6 bytes) 2024-04-24T18:07:41.935 INFO:teuthology.orchestra.run.smithi049.stdout:82 punch from 0x562c388 to 0x563a686, (0xe2fe bytes) 2024-04-24T18:07:41.935 INFO:teuthology.orchestra.run.smithi049.stdout:83 write 0x6172ebf thru 0x6176cea (0x3e2c bytes) 2024-04-24T18:07:41.939 INFO:teuthology.orchestra.run.smithi049.stdout:84 write 0x673d6f6 thru 0x673dcb0 (0x5bb bytes) 2024-04-24T18:07:41.942 INFO:teuthology.orchestra.run.smithi049.stdout:85 read 0x5e5d8c9 thru 0x5e6ba8d (0xe1c5 bytes) 2024-04-24T18:07:41.943 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:41.943 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:41.943 INFO:teuthology.orchestra.run.smithi049.stdout:89 punch from 0xcf68e9b to 0xcf6f26f, (0x63d4 bytes) 2024-04-24T18:07:41.943 INFO:teuthology.orchestra.run.smithi049.stdout:93 read 0xbd6b3e thru 0xbe2aa9 (0xbf6c bytes) 2024-04-24T18:07:41.943 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:41.943 INFO:teuthology.orchestra.run.smithi049.stdout:97 read 0x103df5a thru 0x104940b (0xb4b2 bytes) 2024-04-24T18:07:41.951 INFO:teuthology.orchestra.run.smithi049.stdout:98 trunc from 0xe51fbf9 to 0x7322303 2024-04-24T18:07:41.985 INFO:teuthology.orchestra.run.smithi049.stdout:99 write 0x8e2b41b thru 0x8e338c7 (0x84ad bytes) 2024-04-24T18:07:41.988 INFO:teuthology.orchestra.run.smithi049.stdout:100 read 0x2bc77f thru 0x2c7edc (0xb75e bytes) 2024-04-24T18:07:41.988 INFO:teuthology.orchestra.run.smithi049.stdout:101 read 0x303706 thru 0x3058a4 (0x219f bytes) 2024-04-24T18:07:41.989 INFO:teuthology.orchestra.run.smithi049.stdout:102 writesame 0x7953907 thru 0x7955a69 data_size 0x25(0x2163 bytes) 2024-04-24T18:07:41.991 INFO:teuthology.orchestra.run.smithi049.stdout:103 read 0x78bf54e thru 0x78c2aee (0x35a1 bytes) 2024-04-24T18:07:41.992 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:41.992 INFO:teuthology.orchestra.run.smithi049.stdout:106 write 0x5c30cdd thru 0x5c39538 (0x885c bytes) 2024-04-24T18:07:41.992 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:42.000+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:41.997 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:41.997 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:42.005 INFO:teuthology.orchestra.run.smithi049.stdout:107 write 0xd2f5975 thru 0xd2fb411 (0x5a9d bytes) 2024-04-24T18:07:42.008 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:07:42.009 INFO:teuthology.orchestra.run.smithi049.stdout:108 read 0x484c9e7 thru 0x48502b1 (0x38cb bytes) 2024-04-24T18:07:42.022 INFO:teuthology.orchestra.run.smithi049.stdout:109 trunc from 0xd2fb412 to 0x10df981 2024-04-24T18:07:42.069 INFO:teuthology.orchestra.run.smithi049.stdout:110 read 0xf4815 thru 0x1032ef (0xeadb bytes) 2024-04-24T18:07:42.069 INFO:teuthology.orchestra.run.smithi049.stdout:112 read 0x28ff76 thru 0x298470 (0x84fb bytes) 2024-04-24T18:07:42.085 INFO:teuthology.orchestra.run.smithi049.stdout:113 trunc from 0x10df981 to 0xe3f5740 2024-04-24T18:07:42.092 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:42.101+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:42.093 INFO:teuthology.orchestra.run.smithi049.stdout:114 read 0xaf3e7b4 thru 0xaf42ff7 (0x4844 bytes) 2024-04-24T18:07:42.093 INFO:teuthology.orchestra.run.smithi049.stdout:115 read 0x73516b8 thru 0x735d26f (0xbbb8 bytes) 2024-04-24T18:07:42.093 INFO:teuthology.orchestra.run.smithi049.stdout:116 write 0x7eb14dd thru 0x7eb9ab7 (0x85db bytes) 2024-04-24T18:07:42.101 INFO:teuthology.orchestra.run.smithi049.stdout:117 write 0xe5c563f thru 0xe5c979c (0x415e bytes) 2024-04-24T18:07:42.121 INFO:teuthology.orchestra.run.smithi049.stdout:120 trunc from 0xe5c979d to 0x35f967 2024-04-24T18:07:42.181 INFO:teuthology.orchestra.run.smithi049.stdout:121 write 0xc17b955 thru 0xc17c1cc (0x878 bytes) 2024-04-24T18:07:42.188 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:42.188 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:42.193 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:42.201+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:42.195 INFO:teuthology.orchestra.run.smithi049.stdout:122 trunc from 0xc17c1cd to 0x1a80311 2024-04-24T18:07:42.199 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:07:42.211 INFO:teuthology.orchestra.run.smithi049.stdout:124 writesame 0x31fb091 thru 0x32098a2 data_size 0x41(0xe812 bytes) 2024-04-24T18:07:42.219 INFO:teuthology.orchestra.run.smithi049.stdout:128 writesame 0x60b3045 thru 0x60b63e2 data_size 0x1(0x339e bytes) 2024-04-24T18:07:42.233 INFO:teuthology.orchestra.run.smithi049.stdout:129 writesame 0xd0d92a9 thru 0xd0d9c24 data_size 0x2(0x97c bytes) 2024-04-24T18:07:42.233 INFO:teuthology.orchestra.run.smithi049.stdout:130 writesame 0xd1b51f thru 0xd1e36c data_size 0x1(0x2e4e bytes) 2024-04-24T18:07:42.238 INFO:teuthology.orchestra.run.smithi049.stdout:131 writesame 0x8ce3619 thru 0x8ce8ec1 data_size 0x1(0x58a9 bytes) 2024-04-24T18:07:42.250 INFO:teuthology.orchestra.run.smithi049.stdout:133 write 0xb883d73 thru 0xb886da8 (0x3036 bytes) 2024-04-24T18:07:42.253 INFO:teuthology.orchestra.run.smithi049.stdout:135 writesame 0xa793767 thru 0xa79f9fc data_size 0x1(0xc296 bytes) 2024-04-24T18:07:42.280 INFO:teuthology.orchestra.run.smithi049.stdout:137 writesame 0xa4feb88 thru 0xa4ff0c6 data_size 0x1(0x53f bytes) 2024-04-24T18:07:42.284 INFO:teuthology.orchestra.run.smithi049.stdout:138 read 0xbdba428 thru 0xbdc0bf0 (0x67c9 bytes) 2024-04-24T18:07:42.284 INFO:teuthology.orchestra.run.smithi049.stdout:141 punch from 0x86ec8ad to 0x86f62e1, (0x9a34 bytes) 2024-04-24T18:07:42.284 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.284 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.284 INFO:teuthology.orchestra.run.smithi049.stdout:144 write 0x18de937 thru 0x18df362 (0xa2c bytes) 2024-04-24T18:07:42.287 INFO:teuthology.orchestra.run.smithi049.stdout:146 write 0x1d54611 thru 0x1d60886 (0xc276 bytes) 2024-04-24T18:07:42.291 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.291 INFO:teuthology.orchestra.run.smithi049.stdout:148 read 0x85154a3 thru 0x851a6a9 (0x5207 bytes) 2024-04-24T18:07:42.291 INFO:teuthology.orchestra.run.smithi049.stdout:150 writesame 0x667a0ff thru 0x66852f0 data_size 0x1(0xb1f2 bytes) 2024-04-24T18:07:42.294 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:42.303+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:42.316 INFO:teuthology.orchestra.run.smithi049.stdout:151 write 0xea5fe91 thru 0xea62940 (0x2ab0 bytes) 2024-04-24T18:07:42.320 INFO:teuthology.orchestra.run.smithi049.stdout:153 read 0xc7ae030 thru 0xc7b0e0c (0x2ddd bytes) 2024-04-24T18:07:42.320 INFO:teuthology.orchestra.run.smithi049.stdout:154 write 0x4eaa55f thru 0x4eb0548 (0x5fea bytes) 2024-04-24T18:07:42.323 INFO:teuthology.orchestra.run.smithi049.stdout:155 write 0x91b018a thru 0x91b46ee (0x4565 bytes) 2024-04-24T18:07:42.326 INFO:teuthology.orchestra.run.smithi049.stdout:156 write 0x9c15e06 thru 0x9c18221 (0x241c bytes) 2024-04-24T18:07:42.329 INFO:teuthology.orchestra.run.smithi049.stdout:158 write 0x1af22dc thru 0x1af4036 (0x1d5b bytes) 2024-04-24T18:07:42.331 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.332 INFO:teuthology.orchestra.run.smithi049.stdout:160 write 0x4b59688 thru 0x4b60711 (0x708a bytes) 2024-04-24T18:07:42.336 INFO:teuthology.orchestra.run.smithi049.stdout:161 read 0x8d44974 thru 0x8d44c79 (0x306 bytes) 2024-04-24T18:07:42.336 INFO:teuthology.orchestra.run.smithi049.stdout:163 read 0x728f8b thru 0x737e3e (0xeeb4 bytes) 2024-04-24T18:07:42.336 INFO:teuthology.orchestra.run.smithi049.stdout:165 write 0xbcb7f26 thru 0xbcc492a (0xca05 bytes) 2024-04-24T18:07:42.342 INFO:teuthology.orchestra.run.smithi049.stdout:169 write 0x4cd4f4d thru 0x4cdf48c (0xa540 bytes) 2024-04-24T18:07:42.344 INFO:teuthology.orchestra.run.smithi049.stdout:171 read 0x5edadc1 thru 0x5ee848c (0xd6cc bytes) 2024-04-24T18:07:42.345 INFO:teuthology.orchestra.run.smithi049.stdout:172 punch from 0x534d805 to 0x534fcff, (0x24fa bytes) 2024-04-24T18:07:42.345 INFO:teuthology.orchestra.run.smithi049.stdout:173 write 0x1d0b94f thru 0x1d1425b (0x890d bytes) 2024-04-24T18:07:42.348 INFO:teuthology.orchestra.run.smithi049.stdout:174 read 0x53b5557 thru 0x53bc330 (0x6dda bytes) 2024-04-24T18:07:42.348 INFO:teuthology.orchestra.run.smithi049.stdout:175 writesame 0x5f29a27 thru 0x5f30a03 data_size 0x1(0x6fdd bytes) 2024-04-24T18:07:42.364 INFO:teuthology.orchestra.run.smithi049.stdout:176 writesame 0xb132d91 thru 0xb140fb9 data_size 0x15(0xe229 bytes) 2024-04-24T18:07:42.372 INFO:teuthology.orchestra.run.smithi049.stdout:178 read 0xe7fbafc thru 0xe7fbfa6 (0x4ab bytes) 2024-04-24T18:07:42.376 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:42.377 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:42.382 INFO:teuthology.orchestra.run.smithi049.stdout:179 trunc from 0xea62941 to 0x5a768ac 2024-04-24T18:07:42.388 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:07:42.395 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:42.402+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:42.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:07:42.414 INFO:teuthology.orchestra.run.smithi049.stdout:180 trunc from 0x5a768ac to 0x914ef4b 2024-04-24T18:07:42.420 INFO:teuthology.orchestra.run.smithi049.stdout:181 read 0x187fd3c thru 0x188b4da (0xb79f bytes) 2024-04-24T18:07:42.420 INFO:teuthology.orchestra.run.smithi049.stdout:182 write 0x1b9eb55 thru 0x1ba2219 (0x36c5 bytes) 2024-04-24T18:07:42.432 INFO:teuthology.orchestra.run.smithi049.stdout:183 write 0xc5fbff1 thru 0xc60a57c (0xe58c bytes) 2024-04-24T18:07:42.437 INFO:teuthology.orchestra.run.smithi049.stdout:185 write 0x4109a3a thru 0x4117418 (0xd9df bytes) 2024-04-24T18:07:42.441 INFO:teuthology.orchestra.run.smithi049.stdout:186 trunc from 0xc60a57d to 0xb1c029f 2024-04-24T18:07:42.460 INFO:teuthology.orchestra.run.smithi049.stdout:188 write 0xba7aa9d thru 0xba87bd3 (0xd137 bytes) 2024-04-24T18:07:42.465 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.470 INFO:teuthology.orchestra.run.smithi049.stdout:194 trunc from 0xba87bd4 to 0x6e225dd 2024-04-24T18:07:42.480 INFO:teuthology.orchestra.run.smithi049.stdout:195 read 0x4f5ecb7 thru 0x4f617c4 (0x2b0e bytes) 2024-04-24T18:07:42.480 INFO:teuthology.orchestra.run.smithi049.stdout:198 read 0x6821d41 thru 0x68254b7 (0x3777 bytes) 2024-04-24T18:07:42.480 INFO:teuthology.orchestra.run.smithi049.stdout:199 read 0x15cbd90 thru 0x15cc36e (0x5df bytes) 2024-04-24T18:07:42.480 INFO:teuthology.orchestra.run.smithi049.stdout:200 trunc from 0x6e225dd to 0x722c331 2024-04-24T18:07:42.485 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.493 INFO:teuthology.orchestra.run.smithi049.stdout:202 write 0x9e00cd0 thru 0x9e0b20b (0xa53c bytes) 2024-04-24T18:07:42.495 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:42.503+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:42.496 INFO:teuthology.orchestra.run.smithi049.stdout:205 writesame 0x1968b18 thru 0x1977b1a data_size 0x3(0xf003 bytes) 2024-04-24T18:07:42.508 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.508 INFO:teuthology.orchestra.run.smithi049.stdout:208 read 0x4a2af83 thru 0x4a2e3a8 (0x3426 bytes) 2024-04-24T18:07:42.508 INFO:teuthology.orchestra.run.smithi049.stdout:209 writesame 0x84bcfc7 thru 0x84bf165 data_size 0x13(0x219f bytes) 2024-04-24T18:07:42.511 INFO:teuthology.orchestra.run.smithi049.stdout:210 write 0x1b23416 thru 0x1b2376f (0x35a bytes) 2024-04-24T18:07:42.513 INFO:teuthology.orchestra.run.smithi049.stdout:213 trunc from 0x9e0b20c to 0x89f9282 2024-04-24T18:07:42.523 INFO:teuthology.orchestra.run.smithi049.stdout:214 write 0x45e395f thru 0x45ec440 (0x8ae2 bytes) 2024-04-24T18:07:42.526 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:42.526 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:42.533 INFO:teuthology.orchestra.run.smithi049.stdout:215 trunc from 0x89f9282 to 0x1ba8df3 2024-04-24T18:07:42.537 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:07:42.553 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:07:42.571 INFO:teuthology.orchestra.run.smithi049.stdout:216 writesame 0xb6bb7d6 thru 0xb6c5523 data_size 0x2(0x9d4e bytes) 2024-04-24T18:07:42.572 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:42.576 INFO:teuthology.orchestra.run.smithi049.stdout:218 write 0xbb13e96 thru 0xbb1d076 (0x91e1 bytes) 2024-04-24T18:07:42.581 INFO:teuthology.orchestra.run.smithi049.stdout:219 read 0x20cd59b thru 0x20d4cac (0x7712 bytes) 2024-04-24T18:07:42.581 INFO:teuthology.orchestra.run.smithi049.stdout:220 writesame 0x48b6d69 thru 0x48bf13e data_size 0x7d(0x83d6 bytes) 2024-04-24T18:07:42.585 INFO:teuthology.orchestra.run.smithi049.stdout:221 writesame 0x6b8ac8a thru 0x6b935ec data_size 0x1(0x8963 bytes) 2024-04-24T18:07:42.593 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:42.596 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:42.603+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:42.604 INFO:teuthology.orchestra.run.smithi049.stdout:222 write 0x9fb972c thru 0x9fbd413 (0x3ce8 bytes) 2024-04-24T18:07:42.607 INFO:teuthology.orchestra.run.smithi049.stdout:223 write 0x7aac20c thru 0x7aae8b3 (0x26a8 bytes) 2024-04-24T18:07:42.612 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:42.619 INFO:teuthology.orchestra.run.smithi049.stdout:224 writesame 0xe8ee38c thru 0xe8fe36c data_size 0x37(0xffe1 bytes) 2024-04-24T18:07:42.624 INFO:teuthology.orchestra.run.smithi049.stdout:225 write 0xed9e619 thru 0xed9e7e7 (0x1cf bytes) 2024-04-24T18:07:42.630 INFO:teuthology.orchestra.run.smithi049.stdout:228 read 0x5bc842c thru 0x5bd775c (0xf331 bytes) 2024-04-24T18:07:42.631 INFO:teuthology.orchestra.run.smithi049.stdout:229 write 0x3bf11f2 thru 0x3bf8b78 (0x7987 bytes) 2024-04-24T18:07:42.634 INFO:teuthology.orchestra.run.smithi049.stdout:230 write 0x85dc0c3 thru 0x85eaef9 (0xee37 bytes) 2024-04-24T18:07:42.634 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:42.637 INFO:teuthology.orchestra.run.smithi049.stdout:231 writesame 0xa726e33 thru 0xa735ad7 data_size 0x1(0xeca5 bytes) 2024-04-24T18:07:42.658 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:42.669 INFO:teuthology.orchestra.run.smithi049.stdout:232 write 0x122b6b1 thru 0x123417b (0x8acb bytes) 2024-04-24T18:07:42.679 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:42.684 INFO:teuthology.orchestra.run.smithi049.stdout:234 trunc from 0xed9e7e8 to 0x29e2884 2024-04-24T18:07:42.697 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:42.704+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:42.699 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.699 INFO:teuthology.orchestra.run.smithi049.stdout:239 read 0x241b0f1 thru 0x2423f6e (0x8e7e bytes) 2024-04-24T18:07:42.700 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:42.707 INFO:teuthology.orchestra.run.smithi049.stdout:240 trunc from 0x29e2884 to 0xa5258eb 2024-04-24T18:07:42.710 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:42.710 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:42.710 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:42.713 INFO:teuthology.orchestra.run.smithi049.stdout:242 punch from 0x382994d to 0x382bd8f, (0x2442 bytes) 2024-04-24T18:07:42.713 INFO:teuthology.orchestra.run.smithi049.stdout:243 punch from 0x9ecb193 to 0x9ece20f, (0x307c bytes) 2024-04-24T18:07:42.713 INFO:teuthology.orchestra.run.smithi049.stdout:244 punch from 0x3317c5 to 0x337642, (0x5e7d bytes) 2024-04-24T18:07:42.713 INFO:teuthology.orchestra.run.smithi049.stdout:246 read 0x4e3b5fb thru 0x4e46b2b (0xb531 bytes) 2024-04-24T18:07:42.713 INFO:teuthology.orchestra.run.smithi049.stdout:247 punch from 0xa3a4191 to 0xa3ac670, (0x84df bytes) 2024-04-24T18:07:42.713 INFO:teuthology.orchestra.run.smithi049.stdout:248 write 0x56fd752 thru 0x570a97e (0xd22d bytes) 2024-04-24T18:07:42.717 INFO:teuthology.orchestra.run.smithi049.stdout:249 writesame 0x5ddd8e9 thru 0x5de9b7e data_size 0x1(0xc296 bytes) 2024-04-24T18:07:42.722 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:07:42.723 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:42.724 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:42.741 INFO:teuthology.orchestra.run.smithi049.stdout:251 read 0x480fa01 thru 0x4810224 (0x824 bytes) 2024-04-24T18:07:42.741 INFO:teuthology.orchestra.run.smithi049.stdout:252 writesame 0x999c6a3 thru 0x99a2dae data_size 0xa(0x670c bytes) 2024-04-24T18:07:42.755 INFO:teuthology.orchestra.run.smithi049.stdout:253 writesame 0xa78492b thru 0xa788790 data_size 0x31(0x3e66 bytes) 2024-04-24T18:07:42.756 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.757 INFO:teuthology.orchestra.run.smithi049.stdout:255 trunc from 0xa788791 to 0xc017441 2024-04-24T18:07:42.763 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.763 INFO:teuthology.orchestra.run.smithi049.stdout:260 writesame 0xb8cf56b thru 0xb8d6c8d data_size 0x1(0x7723 bytes) 2024-04-24T18:07:42.763 INFO:teuthology.orchestra.run.smithi049.stdout:261 read 0x40bd6e thru 0x4137eb (0x7a7e bytes) 2024-04-24T18:07:42.768 INFO:teuthology.orchestra.run.smithi049.stdout:262 trunc from 0xc017441 to 0x7c435fc 2024-04-24T18:07:42.789 INFO:teuthology.orchestra.run.smithi049.stdout:263 write 0xaf6f0f2 thru 0xaf6fb68 (0xa77 bytes) 2024-04-24T18:07:42.791 INFO:teuthology.orchestra.run.smithi049.stdout:265 write 0x480c46f thru 0x4814389 (0x7f1b bytes) 2024-04-24T18:07:42.795 INFO:teuthology.orchestra.run.smithi049.stdout:268 read 0x5b76279 thru 0x5b7aca5 (0x4a2d bytes) 2024-04-24T18:07:42.795 INFO:teuthology.orchestra.run.smithi049.stdout:269 write 0x7124bb1 thru 0x7130f54 (0xc3a4 bytes) 2024-04-24T18:07:42.798 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:42.806+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:42.805 INFO:teuthology.orchestra.run.smithi049.stdout:270 write 0xd1b3c09 thru 0xd1b86d1 (0x4ac9 bytes) 2024-04-24T18:07:42.807 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.808 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.808 INFO:teuthology.orchestra.run.smithi049.stdout:274 read 0xcb9225f thru 0xcb95c17 (0x39b9 bytes) 2024-04-24T18:07:42.817 INFO:teuthology.orchestra.run.smithi049.stdout:275 trunc from 0xd1b86d2 to 0x4807e17 2024-04-24T18:07:42.844 INFO:teuthology.orchestra.run.smithi049.stdout:278 write 0xd5161cc thru 0xd52395d (0xd792 bytes) 2024-04-24T18:07:42.848 INFO:teuthology.orchestra.run.smithi049.stdout:279 writesame 0x9a44742 thru 0x9a5305f data_size 0x35(0xe91e bytes) 2024-04-24T18:07:42.852 INFO:teuthology.orchestra.run.smithi049.stdout:280 writesame 0x8b40902 thru 0x8b4d693 data_size 0x62(0xcd92 bytes) 2024-04-24T18:07:42.857 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:42.857 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:42.857 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:42.858 INFO:teuthology.orchestra.run.smithi049.stdout:281 write 0xa14e7e thru 0xa2264c (0xd7cf bytes) 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:282 punch from 0x207e168 to 0x208da0a, (0xf8a2 bytes) 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:283 read 0x750f799 thru 0x7514178 (0x49e0 bytes) 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:284 read 0x9d1160a thru 0x9d1d5f1 (0xbfe8 bytes) 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:285 read 0x6fb9ca3 thru 0x6fc095c (0x6cba bytes) 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:287 punch from 0xbef38ca to 0xbef7b42, (0x4278 bytes) 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:288 punch from 0x79158be to 0x7924168, (0xe8aa bytes) 2024-04-24T18:07:42.861 INFO:teuthology.orchestra.run.smithi049.stdout:289 write 0x2ad0790 thru 0x2ad296b (0x21dc bytes) 2024-04-24T18:07:42.866 INFO:teuthology.orchestra.run.smithi049.stdout:291 write 0xb394e80 thru 0xb3a259b (0xd71c bytes) 2024-04-24T18:07:42.869 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:07:42.875 INFO:teuthology.orchestra.run.smithi049.stdout:293 read 0xa988d0f thru 0xa998052 (0xf344 bytes) 2024-04-24T18:07:42.875 INFO:teuthology.orchestra.run.smithi049.stdout:294 writesame 0x7beff55 thru 0x7bf55c3 data_size 0x1d(0x566f bytes) 2024-04-24T18:07:42.880 INFO:teuthology.orchestra.run.smithi049.stdout:296 read 0x655e06d thru 0x65619f5 (0x3989 bytes) 2024-04-24T18:07:42.880 INFO:teuthology.orchestra.run.smithi049.stdout:297 read 0xcbafe9e thru 0xcbba7bb (0xa91e bytes) 2024-04-24T18:07:42.880 INFO:teuthology.orchestra.run.smithi049.stdout:298 writesame 0x2f0cfa thru 0x2f4843 data_size 0x1(0x3b4a bytes) 2024-04-24T18:07:42.888 INFO:teuthology.orchestra.run.smithi049.stdout:299 clone 3 order 23 su 1048576 sc 3 2024-04-24T18:07:42.913 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:42.922+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:12 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [create,call rbd.create in=9b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e29)", 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:38.298971+0000", 2024-04-24T18:07:42.995 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7027270080000001, 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001668381, 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 12 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298971+0000", 2024-04-24T18:07:42.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298971+0000", 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298970+0000", 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:42.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298976+0000", 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.298987+0000", 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1148999999999999e-05 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299002+0000", 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4087000000000001e-05 2024-04-24T18:07:42.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299042+0000", 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0893999999999998e-05 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299178+0000", 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013531700000000001 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:07:42.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.299400+0000", 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022242299999999999 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300349+0000", 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00094871899999999999 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300562+0000", 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000212914 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300599+0000", 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909000000000002e-05 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:38.300640+0000", 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1075999999999998e-05 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:96 4.12 4:483ae98a:::rbd_header.11741fd60f53:head [call rbd.snapshot_add in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:40.293731+0000", 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 2.7079671059999999, 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001810056, 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 96 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293731+0000", 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293731+0000", 2024-04-24T18:07:43.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293730+0000", 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293738+0000", 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3639999999999999e-06 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293753+0000", 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4926000000000001e-05 2024-04-24T18:07:43.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293761+0000", 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4780000000000002e-06 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293807+0000", 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6179e-05 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.293905+0000", 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8591999999999994e-05 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.294299+0000", 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000393805 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.294764+0000", 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00046436399999999999 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.295456+0000", 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00069246900000000003 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.295509+0000", 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2782999999999999e-05 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.295541+0000", 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.26e-05 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:148 4.1f 4:f831ff65:::rbd_header.117480dfac0d:head [call rbd.parent_attach in=16b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e31)", 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:40.333076+0000", 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 2.6686221830000001, 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0027623349999999999, 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 148 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333076+0000", 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333076+0000", 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333076+0000", 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333080+0000", 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0319999999999997e-06 2024-04-24T18:07:43.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333090+0000", 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9550000000000004e-06 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333094+0000", 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5340000000000001e-06 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333117+0000", 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2912999999999999e-05 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333164+0000", 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6907999999999999e-05 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.333304+0000", 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00014025999999999999 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.335557+0000", 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002252315 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.335775+0000", 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00021799300000000001 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.335804+0000", 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9169000000000001e-05 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:40.335839+0000", 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4897999999999997e-05 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1852 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.722372+0000", 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.27932605599999999, 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020877119999999998, 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1852 2024-04-24T18:07:43.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722372+0000", 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722372+0000", 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722371+0000", 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:43.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722377+0000", 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2999999999999998e-06 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722390+0000", 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2408e-05 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722395+0000", 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3329999999999996e-06 2024-04-24T18:07:43.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722460+0000", 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4565000000000003e-05 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722518+0000", 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8103000000000003e-05 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722751+0000", 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00023361700000000001 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724151+0000", 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001399689 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724391+0000", 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024037200000000001 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724423+0000", 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1211000000000003e-05 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724460+0000", 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7568999999999999e-05 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1853 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.722403+0000", 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.27929548100000001, 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0023031039999999998, 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.017 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1853 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722403+0000", 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722403+0000", 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722403+0000", 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722405+0000", 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8200000000000001e-06 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722409+0000", 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8700000000000002e-06 2024-04-24T18:07:43.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.722413+0000", 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1429999999999998e-06 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.723060+0000", 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000647234 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.723113+0000", 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2988000000000002e-05 2024-04-24T18:07:43.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.723329+0000", 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00021577200000000001 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724474+0000", 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001145557 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724650+0000", 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00017565199999999999 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724676+0000", 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5967999999999999e-05 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.724706+0000", 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0406e-05 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.022 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1857 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.726333+0000", 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.27536586499999999, 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002140032, 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1857 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.726333+0000", 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.726333+0000", 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.726332+0000", 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.726335+0000", 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6359999999999998e-06 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.726341+0000", 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7300000000000002e-06 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.726344+0000", 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.727019+0000", 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00067411000000000001 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.727108+0000", 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9839999999999994e-05 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.727468+0000", 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00035989300000000002 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.728205+0000", 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00073724499999999998 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.728417+0000", 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000211339 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.728445+0000", 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8396000000000001e-05 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.728473+0000", 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7402e-05 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1860 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.750428+0000", 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.25127094799999999, 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017268159999999999, 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1860 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.028 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750428+0000", 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750428+0000", 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750426+0000", 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:43.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750437+0000", 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1395e-05 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750454+0000", 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7452000000000002e-05 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750462+0000", 2024-04-24T18:07:43.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7219999999999999e-06 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750505+0000", 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2756000000000002e-05 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750579+0000", 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4519999999999998e-05 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750826+0000", 2024-04-24T18:07:43.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024683000000000002 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.751985+0000", 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011591920000000001 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.752065+0000", 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9771000000000006e-05 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.752114+0000", 2024-04-24T18:07:43.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8945e-05 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.752154+0000", 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0139000000000003e-05 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1861 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.750474+0000", 2024-04-24T18:07:43.033 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.251224634, 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042460909999999996, 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1861 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750474+0000", 2024-04-24T18:07:43.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750474+0000", 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750473+0000", 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750477+0000", 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7890000000000001e-06 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750483+0000", 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1670000000000004e-06 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.750487+0000", 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.751252+0000", 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000764852 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.751341+0000", 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8945999999999998e-05 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.751689+0000", 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00034843599999999998 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.752511+0000", 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00082121099999999997 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.754644+0000", 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021336810000000001 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.754685+0000", 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0771e-05 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.754720+0000", 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4817000000000002e-05 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1864 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_resize in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.758189+0000", 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.24350917599999999, 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020288160000000001, 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.039 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1864 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758189+0000", 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758189+0000", 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758188+0000", 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758195+0000", 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.002e-06 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758209+0000", 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3552e-05 2024-04-24T18:07:43.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758217+0000", 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5099999999999998e-06 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758254+0000", 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6582000000000003e-05 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758325+0000", 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0856000000000001e-05 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.758585+0000", 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00026041399999999999 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.759838+0000", 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012531560000000001 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.760134+0000", 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00029638899999999998 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.760182+0000", 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7895000000000001e-05 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.760218+0000", 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5713999999999997e-05 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:1884 4.e e33/26 4:7492a4e9:::rbd_data.1174cab6075.00000000000000a3:head v 33'10, mlcod=33'8)", 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.779097+0000", 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.22260168599999999, 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0022847760000000001, 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779097+0000", 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779097+0000", 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779096+0000", 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779103+0000", 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9e-06 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779115+0000", 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.183e-05 2024-04-24T18:07:43.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779121+0000", 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.879e-06 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779156+0000", 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4983e-05 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.779179+0000", 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3589000000000001e-05 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781343+0000", 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021637850000000001 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781382+0000", 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.854e-05 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1886 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.781665+0000", 2024-04-24T18:07:43.048 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.22003377399999999, 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002524503, 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1886 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781665+0000", 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781665+0000", 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781663+0000", 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781670+0000", 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5969999999999997e-06 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781684+0000", 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4052e-05 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781690+0000", 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.579e-06 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781725+0000", 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4808999999999998e-05 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.781781+0000", 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5945000000000001e-05 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.782005+0000", 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022360199999999999 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.783937+0000", 2024-04-24T18:07:43.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019324959999999999 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.784102+0000", 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000164542 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.784150+0000", 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8185e-05 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.784189+0000", 2024-04-24T18:07:43.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9150000000000003e-05 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1950 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.853558+0000", 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.14814008200000001, 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001672467, 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.054 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1950 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853558+0000", 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853558+0000", 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853557+0000", 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853564+0000", 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853577+0000", 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3784000000000001e-05 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853609+0000", 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1377000000000001e-05 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853643+0000", 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3958999999999998e-05 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853695+0000", 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1940000000000001e-05 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853907+0000", 2024-04-24T18:07:43.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000212544 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.854938+0000", 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001030932 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.855179+0000", 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024072699999999999 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.855209+0000", 2024-04-24T18:07:43.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0414999999999999e-05 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.855231+0000", 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1505000000000002e-05 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1951 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.853619+0000", 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.14807937700000001, 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020638990000000001, 2024-04-24T18:07:43.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1951 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853619+0000", 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853619+0000", 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853619+0000", 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853623+0000", 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3120000000000004e-06 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.061 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853630+0000", 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.853635+0000", 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4870000000000004e-06 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.854287+0000", 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00065215100000000001 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.854372+0000", 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5198999999999995e-05 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.854630+0000", 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000258142 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.855530+0000", 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089985200000000003 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.855595+0000", 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5133000000000001e-05 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.855647+0000", 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1232e-05 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.855683+0000", 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6436000000000001e-05 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1956 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.861719+0000", 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.13997910299999999, 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029464209999999998, 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1956 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.065 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861719+0000", 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861719+0000", 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861719+0000", 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861724+0000", 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9099999999999996e-06 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861734+0000", 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0239e-05 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861738+0000", 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.459e-06 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861777+0000", 2024-04-24T18:07:43.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8747e-05 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.861861+0000", 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3784999999999998e-05 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.862222+0000", 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000360956 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864387+0000", 2024-04-24T18:07:43.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021657489999999998 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864618+0000", 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000230724 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864645+0000", 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7123000000000001e-05 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864666+0000", 2024-04-24T18:07:43.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0458999999999998e-05 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1957 4.5 4:a76d3376:::rbd_data.1174cab6075.000000000000008b:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.864846+0000", 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.13685251600000001, 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002727101, 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1957 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864846+0000", 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864846+0000", 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864845+0000", 2024-04-24T18:07:43.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864849+0000", 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864859+0000", 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0051000000000001e-05 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864864+0000", 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5959999999999997e-06 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864886+0000", 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2382e-05 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.864957+0000", 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0957999999999994e-05 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.865092+0000", 2024-04-24T18:07:43.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013442200000000001 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.867262+0000", 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00217067 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.867513+0000", 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00025036500000000001 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.867548+0000", 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5346999999999999e-05 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.867573+0000", 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4848999999999999e-05 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1960 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.869952+0000", 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.13174633499999999, 2024-04-24T18:07:43.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038317939999999999, 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1960 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.869952+0000", 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.869952+0000", 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.869951+0000", 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.869957+0000", 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4060000000000004e-06 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.869967+0000", 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0233e-05 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.869972+0000", 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6539999999999998e-06 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.870000+0000", 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8605000000000001e-05 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.870059+0000", 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259000000000003e-05 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.870269+0000", 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000209497 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.870944+0000", 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00067487999999999999 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.873728+0000", 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002783752 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.873760+0000", 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2530999999999997e-05 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.873784+0000", 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3842e-05 2024-04-24T18:07:43.080 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1962 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.876016+0000", 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.12568199799999999, 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017283120000000001, 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1962 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876016+0000", 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876016+0000", 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876016+0000", 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876020+0000", 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.048e-06 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876028+0000", 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3489999999999992e-06 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876032+0000", 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8650000000000003e-06 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876068+0000", 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6149999999999998e-05 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876134+0000", 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6215000000000002e-05 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876379+0000", 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024497800000000003 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.877642+0000", 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001262946 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.877673+0000", 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0351000000000001e-05 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.877717+0000", 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4648999999999999e-05 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.877745+0000", 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7508000000000002e-05 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1963 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.876042+0000", 2024-04-24T18:07:43.086 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.12565611400000001, 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002175181, 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1963 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876042+0000", 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876042+0000", 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876042+0000", 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876044+0000", 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2620000000000001e-06 2024-04-24T18:07:43.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876049+0000", 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876053+0000", 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876805+0000", 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00075254599999999999 2024-04-24T18:07:43.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.876893+0000", 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7174000000000003e-05 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.877236+0000", 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00034362900000000003 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.877886+0000", 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00064931500000000003 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878138+0000", 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000252544 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878183+0000", 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4811999999999997e-05 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878218+0000", 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4650000000000002e-05 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:1965 4.18 e33/26 4:1d893d51:::rbd_data.1174cab6075.00000000000000b1:head v 33'7, mlcod=32'5)", 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.878614+0000", 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.12308454200000001, 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025551129999999999, 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878614+0000", 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878614+0000", 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878613+0000", 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878626+0000", 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2687999999999999e-05 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878637+0000", 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0725e-05 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878642+0000", 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.558e-06 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878678+0000", 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5571000000000002e-05 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.878690+0000", 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2639e-05 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.881134+0000", 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024430939999999998 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.881169+0000", 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5494999999999998e-05 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:1966 4.5 4:a4ff1bae:::rbd_object_map.1174cab6075:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:42.884180+0000", 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.117517973, 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002851316, 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:07:43.096 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 1966 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884180+0000", 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884180+0000", 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884179+0000", 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884186+0000", 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3570000000000003e-06 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884197+0000", 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1576999999999999e-05 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884202+0000", 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.955e-06 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884238+0000", 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5382000000000001e-05 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884298+0000", 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0720000000000001e-05 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 4", 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.884524+0000", 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000225176 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.886702+0000", 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002177993 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.886975+0000", 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000273422 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.887006+0000", 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0913e-05 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:42.887032+0000", 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5931e-05 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.101 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:43.103 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:43.110+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:43.103 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:07:43.204 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:43.211+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:43.269 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:43.269 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:43.269 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:43.279 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:07:43.322 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:43.331+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:43.404 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:43.404 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:43.404 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:43.405 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:43.413+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:43.414 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:07:43.506 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:43.514+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973758368999999, 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:07:43.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:07:43.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:07:43.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:07:43.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.589 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973689317000002, 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:07:43.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:07:43.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:07:43.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:07:43.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:07:43.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973664901999999, 2024-04-24T18:07:43.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:07:43.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973641752999999, 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.604 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:07:43.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:07:43.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973561185000001, 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.609 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.610 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.612 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:07:43.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:07:43.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973536374999998, 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.615 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:43.616 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:07:43.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:07:43.618 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.619 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.620 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973516263, 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.621 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.622 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.623 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.624 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:07:43.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973445600000002, 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.626 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.627 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:07:43.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:07:43.629 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.630 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.973433827000001, 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:07:43.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:07:43.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:07:43.637 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.553450465000001, 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0070057069999999999, 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.638 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 132 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036533+0000", 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036535+0000", 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036539+0000", 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3459999999999998e-06 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036541+0000", 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9520000000000001e-06 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037943+0000", 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001402401 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038021+0000", 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8071000000000005e-05 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.641 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038150+0000", 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012869199999999999 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.039054+0000", 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00090387400000000004 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043447+0000", 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043929110000000002 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043497+0000", 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0420999999999997e-05 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043539+0000", 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2255000000000001e-05 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.643 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.553148215, 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0068998769999999996, 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036839+0000", 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9739999999999998e-06 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036844+0000", 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3079999999999998e-06 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036846+0000", 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4099999999999998e-06 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040072+0000", 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003225805 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040118+0000", 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6026999999999998e-05 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040191+0000", 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2983000000000002e-05 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042013+0000", 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001822118 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.647 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043702+0000", 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0016890329999999999 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043721+0000", 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8754999999999999e-05 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043736+0000", 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4725e-05 2024-04-24T18:07:43.648 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.553111997999999, 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.649 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:07:43.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:07:43.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.553064815999999, 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:07:43.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:43.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:07:43.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:07:43.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:07:43.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.553039694999999, 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:07:43.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:07:43.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:07:43.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.669 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.552987560999998, 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:07:43.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:43.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:07:43.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:07:43.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:07:43.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.552924097000002, 2024-04-24T18:07:43.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:07:43.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:07:43.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.041753797999998, 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073643390000000001, 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.681 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 156 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548239+0000", 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8899999999999996e-06 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548246+0000", 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9370000000000004e-06 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548259+0000", 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549586+0000", 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001326743 2024-04-24T18:07:43.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549641+0000", 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5856999999999997e-05 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549713+0000", 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1970000000000004e-05 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550899+0000", 2024-04-24T18:07:43.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001185567 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555532+0000", 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0046325760000000002 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555573+0000", 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1730000000000002e-05 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555595+0000", 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1324000000000001e-05 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:43.687 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.041718424999999, 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:07:43.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:07:43.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:07:43.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.041693398, 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:07:43.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:07:43.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.700 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.041614160999998, 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:43.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:43.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:07:43.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:43.706 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:43.707 INFO:teuthology.orchestra.run.smithi049.stdout:flattening image image_client.0-clone2 2024-04-24T18:07:43.708 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:07:43.709 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:43.715+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:43.755 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #1, image image_client.0-clone1 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent2 2024-04-24T18:07:43.810 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:43.817+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:43.850 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:43.850 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:43.850 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:43.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:07:43.911 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:43.919+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:43.932 INFO:teuthology.orchestra.run.smithi049.stdout:300 read 0xd0865ce thru 0xd08e6ee (0x8121 bytes) 2024-04-24T18:07:43.933 INFO:teuthology.orchestra.run.smithi049.stdout:304 write 0x320f7d1 thru 0x3210189 (0x9b9 bytes) 2024-04-24T18:07:43.947 INFO:teuthology.orchestra.run.smithi049.stdout:306 trunc from 0xd52395e to 0x733fb25 2024-04-24T18:07:43.956 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:43.956 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:43.956 INFO:teuthology.orchestra.run.smithi049.stdout:310 read 0x415d67c thru 0x4166966 (0x92eb bytes) 2024-04-24T18:07:43.956 INFO:teuthology.orchestra.run.smithi049.stdout:311 read 0x328cf85 thru 0x329735f (0xa3db bytes) 2024-04-24T18:07:43.969 INFO:teuthology.orchestra.run.smithi049.stdout:312 writesame 0xdadc5c6 thru 0xdade4b6 data_size 0x1(0x1ef1 bytes) 2024-04-24T18:07:43.969 INFO:teuthology.orchestra.run.smithi049.stdout:313 punch from 0x4b3f838 to 0x4b446c5, (0x4e8d bytes) 2024-04-24T18:07:43.983 INFO:teuthology.orchestra.run.smithi049.stdout:314 trunc from 0xdade4b7 to 0x1544181 2024-04-24T18:07:44.006 INFO:teuthology.orchestra.run.smithi049.stdout:315 read 0xb8ee64 thru 0xb8f92a (0xac7 bytes) 2024-04-24T18:07:44.006 INFO:teuthology.orchestra.run.smithi049.stdout:317 punch from 0xd2357d to 0xd288c6, (0x5349 bytes) 2024-04-24T18:07:44.011 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:44.020+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:44.016 INFO:teuthology.orchestra.run.smithi049.stdout:318 trunc from 0x1544181 to 0xa88ed92 2024-04-24T18:07:44.021 INFO:teuthology.orchestra.run.smithi049.stdout:319 read 0x6a4e513 thru 0x6a50138 (0x1c26 bytes) 2024-04-24T18:07:44.021 INFO:teuthology.orchestra.run.smithi049.stdout:320 write 0x2287c36 thru 0x228951e (0x18e9 bytes) 2024-04-24T18:07:44.024 INFO:teuthology.orchestra.run.smithi049.stdout:321 write 0x44f9a1a thru 0x44fa248 (0x82f bytes) 2024-04-24T18:07:44.028 INFO:teuthology.orchestra.run.smithi049.stdout:322 trunc from 0xa88ed92 to 0x922d83c 2024-04-24T18:07:44.036 INFO:teuthology.orchestra.run.smithi049.stdout:323 trunc from 0x922d83c to 0xab90054 2024-04-24T18:07:44.042 INFO:teuthology.orchestra.run.smithi049.stdout:324 write 0x3ab6eb7 thru 0x3ac3aa1 (0xcbeb bytes) 2024-04-24T18:07:44.045 INFO:teuthology.orchestra.run.smithi049.stdout:326 read 0x3328e35 thru 0x3335eca (0xd096 bytes) 2024-04-24T18:07:44.045 INFO:teuthology.orchestra.run.smithi049.stdout:327 punch from 0x48d27f9 to 0x48d39b0, (0x11b7 bytes) 2024-04-24T18:07:44.049 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:44.049 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:44.049 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:44.053 INFO:teuthology.orchestra.run.smithi049.stdout:328 writesame 0xd34526d thru 0xd345acf data_size 0x1(0x863 bytes) 2024-04-24T18:07:44.053 INFO:teuthology.orchestra.run.smithi049.stdout:329 read 0xc6a70db thru 0xc6b0658 (0x957e bytes) 2024-04-24T18:07:44.054 INFO:teuthology.orchestra.run.smithi049.stdout:330 read 0xa2e97a7 thru 0xa2f3e83 (0xa6dd bytes) 2024-04-24T18:07:44.054 INFO:teuthology.orchestra.run.smithi049.stdout:331 write 0x5310eef thru 0x5318cf9 (0x7e0b bytes) 2024-04-24T18:07:44.057 INFO:teuthology.orchestra.run.smithi049.stdout:332 writesame 0xb78c73e thru 0xb795074 data_size 0x9(0x8937 bytes) 2024-04-24T18:07:44.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:07:44.062 INFO:teuthology.orchestra.run.smithi049.stdout:333 write 0x540f735 thru 0x5418c9b (0x9567 bytes) 2024-04-24T18:07:44.065 INFO:teuthology.orchestra.run.smithi049.stdout:334 punch from 0x111a9bc to 0x1122102, (0x7746 bytes) 2024-04-24T18:07:44.065 INFO:teuthology.orchestra.run.smithi049.stdout:335 read 0x47ce4e4 thru 0x47d4ee3 (0x6a00 bytes) 2024-04-24T18:07:44.065 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:44.065 INFO:teuthology.orchestra.run.smithi049.stdout:339 clone 4 order 22 su 2097152 sc 14 2024-04-24T18:07:44.112 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:44.121+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "MOSDPGPush(2.0 29/28 [PushOp(2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 17'116, data_included: [0~262688], data_size: 262688, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@17'116, size: 262688, copy_subset: [0~262688], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:262688, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:38.295504+0000", 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.9052974320000002, 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098017500000000001, 2024-04-24T18:07:44.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295504+0000", 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295504+0000", 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295503+0000", 2024-04-24T18:07:44.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295787+0000", 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028404199999999998 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295809+0000", 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1637999999999998e-05 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295828+0000", 2024-04-24T18:07:44.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9018999999999999e-05 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295898+0000", 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0375000000000003e-05 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.295912+0000", 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3162999999999999e-05 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.296484+0000", 2024-04-24T18:07:44.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00057271500000000001 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "MOSDPGPush(2.0 29/28 [PushOp(2:30306672:devicehealth::main.db.0000000000000000:head, version: 17'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(2:30306672:devicehealth::main.db.0000000000000000:head@17'117, size: 327680, copy_subset: [0~327680], clone_subset: {}, snapset: 0=[]:{}, object_exist: 0), after_progress: ObjectRecoveryProgress(!first, data_recovered_to:327680, data_complete:true, omap_recovered_to:, omap_complete:true, error:false), before_progress: ObjectRecoveryProgress(first, data_recovered_to:0, data_complete:false, omap_recovered_to:, omap_complete:false, error:false))])", 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:38.298100+0000", 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.9027015929999997, 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00078320799999999997, 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298100+0000", 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298100+0000", 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298099+0000", 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298400+0000", 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030102999999999998 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298415+0000", 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4348999999999999e-05 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298428+0000", 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3059e-05 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298474+0000", 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6196999999999998e-05 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298479+0000", 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.775e-06 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:38.298883+0000", 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00040442400000000002 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:67 4.7 e30/27 4:e2fa8be3:::rbd_data.11741fd60f53.000000000000002d:head v 30'1, mlcod=0'0)", 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:40.068677+0000", 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.1321244620000002, 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00087456399999999998, 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068677+0000", 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068677+0000", 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068677+0000", 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068774+0000", 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7040999999999995e-05 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068788+0000", 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4612e-05 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068798+0000", 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0147e-05 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068843+0000", 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4555999999999998e-05 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.068858+0000", 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5396999999999998e-05 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.069513+0000", 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00065485399999999996 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.069552+0000", 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8324000000000001e-05 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:79 4.7 e30/27 4:e2fa8be3:::rbd_data.11741fd60f53.000000000000002d:head v 30'2, mlcod=0'0)", 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:40.082487+0000", 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.118314475, 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00078901700000000004, 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082487+0000", 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082487+0000", 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082485+0000", 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999976 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082502+0000", 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7075000000000002e-05 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082525+0000", 2024-04-24T18:07:44.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3141000000000001e-05 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082537+0000", 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2510999999999999e-05 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082608+0000", 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0741000000000006e-05 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.082628+0000", 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9587999999999999e-05 2024-04-24T18:07:44.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.083236+0000", 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00060801300000000002 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:40.083276+0000", 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0380000000000003e-05 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:1794 4.16 4:69041f20:::rbd_data.1174cab6075.000000000000007e:head [set-alloc-hint object_size 1048576 write_size 1048576,write 180748~9896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:44.209 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.618439+0000", 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.5823620709999999, 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001398685, 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 1794 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618439+0000", 2024-04-24T18:07:44.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618439+0000", 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618437+0000", 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618455+0000", 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7112000000000002e-05 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618476+0000", 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1679999999999999e-05 2024-04-24T18:07:44.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618486+0000", 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6350000000000001e-06 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618535+0000", 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8971000000000003e-05 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618643+0000", 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000108349 2024-04-24T18:07:44.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.618752+0000", 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000109237 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.619548+0000", 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000795329 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:44.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.619777+0000", 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000229674 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.619812+0000", 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4096e-05 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.619838+0000", 2024-04-24T18:07:44.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6562000000000001e-05 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:1818 4.16 4:69041f20:::rbd_data.1174cab6075.000000000000007e:head [write 917504~44794] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.643233+0000", 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.5575689079999999, 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014116350000000001, 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:44.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 1818 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643233+0000", 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643233+0000", 2024-04-24T18:07:44.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643230+0000", 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999976 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643254+0000", 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3813999999999999e-05 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643272+0000", 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7779e-05 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643279+0000", 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0350000000000004e-06 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643336+0000", 2024-04-24T18:07:44.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7182999999999998e-05 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643432+0000", 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5989000000000005e-05 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.643595+0000", 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000163096 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.644373+0000", 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077817800000000003 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.644592+0000", 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000218872 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.644621+0000", 2024-04-24T18:07:44.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9022000000000001e-05 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.644644+0000", 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2979000000000001e-05 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.225 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:1821 4.1e 4:7e3169c0:::rbd_data.1174cab6075.00000000000000a7:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.648686+0000", 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.552115482, 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.029350831000000001, 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:44.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 1821 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648686+0000", 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648686+0000", 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648685+0000", 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648692+0000", 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4340000000000003e-06 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648707+0000", 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4857e-05 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648714+0000", 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1119999999999998e-06 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648761+0000", 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6959999999999998e-05 2024-04-24T18:07:44.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648847+0000", 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5464999999999996e-05 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.653108+0000", 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042610640000000002 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.677921+0000", 2024-04-24T18:07:44.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.024812873999999999 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.677964+0000", 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3367e-05 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.678007+0000", 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2809000000000003e-05 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.678037+0000", 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0170000000000001e-05 2024-04-24T18:07:44.231 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop_reply(client.4468.0:1821 4.1e e33/26 ondisk, result = 0)", 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.669539+0000", 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.5312623969999999, 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085041209999999999, 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.669539+0000", 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.669539+0000", 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.669538+0000", 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.669546+0000", 2024-04-24T18:07:44.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1289999999999996e-06 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.669567+0000", 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1590000000000002e-05 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.669578+0000", 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0331e-05 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.677952+0000", 2024-04-24T18:07:44.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0083744370000000002 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.677959+0000", 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3069999999999997e-06 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.678043+0000", 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3794000000000004e-05 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.235 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:1828 4.16 4:69041f20:::rbd_data.1174cab6075.000000000000007e:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.696027+0000", 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.5047744789999999, 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001758616, 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 1828 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696027+0000", 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696027+0000", 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696025+0000", 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999976 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696035+0000", 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0321e-05 2024-04-24T18:07:44.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696055+0000", 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0205000000000001e-05 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696064+0000", 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9090000000000006e-06 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696132+0000", 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7615999999999998e-05 2024-04-24T18:07:44.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696239+0000", 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010725700000000001 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696343+0000", 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010425300000000001 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697311+0000", 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096805199999999995 2024-04-24T18:07:44.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697709+0000", 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00039801099999999998 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697749+0000", 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9128999999999998e-05 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697786+0000", 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7088999999999997e-05 2024-04-24T18:07:44.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:1830 4.1e 4:7e3169c0:::rbd_data.1174cab6075.00000000000000a7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.696082+0000", 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.5047191390000001, 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00158998, 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 1830 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.241 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696082+0000", 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696082+0000", 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696082+0000", 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696086+0000", 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8870000000000002e-06 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696108+0000", 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2096e-05 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696114+0000", 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3260000000000001e-06 2024-04-24T18:07:44.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696174+0000", 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0136999999999997e-05 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696266+0000", 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1188999999999997e-05 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.696393+0000", 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000127112 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697528+0000", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011356929999999999 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697581+0000", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2741000000000002e-05 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697627+0000", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6421999999999997e-05 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.697672+0000", 2024-04-24T18:07:44.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4882000000000002e-05 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:1906 4.7 e33/27 4:e7314dfe:::rbd_data.1174cab6075.00000000000000ad:head v 33'12, mlcod=32'10)", 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.799867+0000", 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.4009346110000001, 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00082134899999999997, 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799867+0000", 2024-04-24T18:07:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799867+0000", 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799866+0000", 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799880+0000", 2024-04-24T18:07:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.415e-05 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799893+0000", 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2830000000000001e-05 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799901+0000", 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7400000000000004e-06 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799961+0000", 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9787000000000002e-05 2024-04-24T18:07:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.799987+0000", 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6911999999999999e-05 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.800657+0000", 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000670047 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.800688+0000", 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0775999999999998e-05 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:1928 4.7 e33/27 4:e7314dfe:::rbd_data.1174cab6075.00000000000000ad:head v 33'13, mlcod=32'11)", 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.829237+0000", 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.3715642720000001, 2024-04-24T18:07:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00073535199999999999, 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829237+0000", 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829237+0000", 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829235+0000", 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829248+0000", 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2488e-05 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829267+0000", 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8998999999999999e-05 2024-04-24T18:07:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829274+0000", 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5299999999999999e-06 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829344+0000", 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9572000000000004e-05 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829370+0000", 2024-04-24T18:07:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6245e-05 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829937+0000", 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00056693800000000001 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.829973+0000", 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5590000000000003e-05 2024-04-24T18:07:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:1958 4.7 e33/27 4:e7551a2e:::rbd_data.1174cab6075.0000000000000005:head v 33'14, mlcod=33'12)", 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.868637+0000", 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.332164356, 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00079373699999999998, 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868637+0000", 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868637+0000", 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868636+0000", 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868669+0000", 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2546999999999999e-05 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868685+0000", 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6254000000000001e-05 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868690+0000", 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4600000000000002e-06 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868727+0000", 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6955000000000001e-05 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.868743+0000", 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5903e-05 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.869389+0000", 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00064539100000000004 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.869431+0000", 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2364000000000003e-05 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:2022 4.7 e34/27 4:e157aeb7:::rbd_header.1174b1b97c76:head v 34'15, mlcod=33'13)", 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:43.600505+0000", 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.60029666199999998, 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00068433399999999998, 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600505+0000", 2024-04-24T18:07:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600505+0000", 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600505+0000", 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600513+0000", 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9079999999999995e-06 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600524+0000", 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1895999999999999e-05 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600532+0000", 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2080000000000003e-06 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600579+0000", 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7636999999999998e-05 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.600595+0000", 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5953000000000001e-05 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.601152+0000", 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000557221 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.601189+0000", 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6709999999999999e-05 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:2095 4.7 e34/27 4:e157aeb7:::rbd_header.1174b1b97c76:head v 34'20, mlcod=34'18)", 2024-04-24T18:07:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:43.756417+0000", 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.44438453500000002, 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00078043800000000005, 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756417+0000", 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756417+0000", 2024-04-24T18:07:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756416+0000", 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756428+0000", 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2328e-05 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756443+0000", 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5389000000000001e-05 2024-04-24T18:07:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756450+0000", 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.178e-06 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756491+0000", 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1461000000000002e-05 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.756509+0000", 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7929000000000001e-05 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.757163+0000", 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00065405400000000005 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.757197+0000", 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4369999999999998e-05 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:2137 4.7 e34/27 4:e157aeb7:::rbd_header.1174b1b97c76:head v 34'21, mlcod=34'19)", 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:43.942887+0000", 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.25791500699999997, 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00076175500000000003, 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.942887+0000", 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.942887+0000", 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.942885+0000", 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.942900+0000", 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4975e-05 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.942919+0000", 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9225999999999999e-05 2024-04-24T18:07:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.942930+0000", 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0841e-05 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.942990+0000", 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0103999999999999e-05 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.943020+0000", 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9833999999999999e-05 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.943606+0000", 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00058575200000000004 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.943648+0000", 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2590999999999999e-05 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:2164 4.7 e34/27 4:e157aeb7:::rbd_header.1174b1b97c76:head v 34'23, mlcod=34'21)", 2024-04-24T18:07:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:43.975506+0000", 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.22529581500000001, 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00060259400000000002, 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975506+0000", 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975506+0000", 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975505+0000", 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975523+0000", 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8839e-05 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975539+0000", 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5610000000000001e-05 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975547+0000", 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5279999999999998e-06 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975585+0000", 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8040999999999997e-05 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.975602+0000", 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7025e-05 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.976065+0000", 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00046300399999999999 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:43.976108+0000", 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3646999999999997e-05 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:2362 4.7 e34/27 4:e157aeb7:::rbd_header.1174b1b97c76:head v 34'26, mlcod=34'24)", 2024-04-24T18:07:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:44.038423+0000", 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.16237828000000001, 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00072050900000000004, 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038423+0000", 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038423+0000", 2024-04-24T18:07:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038422+0000", 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038447+0000", 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5452e-05 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038466+0000", 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8987e-05 2024-04-24T18:07:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038474+0000", 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.259e-06 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038523+0000", 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8454000000000001e-05 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.038541+0000", 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8164000000000001e-05 2024-04-24T18:07:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.039114+0000", 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00057269000000000005 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.039144+0000", 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0028000000000001e-05 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:2392 4.3 4:c28f2cd5:::rbd_data.1174b1b97c76.000000000000000b:head [set-alloc-hint object_size 8388608 write_size 8388608,write 3215087~32267] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:44.064363+0000", 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.13643807999999999, 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019611170000000001, 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 2392 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064363+0000", 2024-04-24T18:07:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064363+0000", 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064362+0000", 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064379+0000", 2024-04-24T18:07:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7567e-05 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064394+0000", 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5281000000000002e-05 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064401+0000", 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4110000000000001e-06 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064456+0000", 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5127999999999998e-05 2024-04-24T18:07:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064612+0000", 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015578900000000001 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.064777+0000", 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00016579499999999999 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.065623+0000", 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084590900000000005 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.066236+0000", 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061225099999999996 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.066286+0000", 2024-04-24T18:07:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0247000000000002e-05 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.066325+0000", 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8648e-05 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:2396 4.3 4:c2114a66:::rbd_data.1174b1b97c76.0000000000000009:head [set-alloc-hint object_size 8388608 write_size 8388608,write 4257589~38247] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e34)", 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:44.072100+0000", 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.12870143100000001, 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015249790000000001, 2024-04-24T18:07:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 2396 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072100+0000", 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072100+0000", 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072098+0000", 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072121+0000", 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2994e-05 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072138+0000", 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6300999999999999e-05 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072145+0000", 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5020000000000003e-06 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072191+0000", 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5998999999999998e-05 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072287+0000", 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6057000000000005e-05 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.072379+0000", 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2264999999999998e-05 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.073043+0000", 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00066356599999999998 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.073556+0000", 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000512883 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.073586+0000", 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0029999999999999e-05 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:44.073625+0000", 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9212000000000003e-05 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.295 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:44.297 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:07:44.297 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:44.304+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:44.397 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:44.404+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:44.426 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:44.426 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:44.427 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:44.427 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:44.437 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:07:44.521 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:44.529+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:44.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:07:44.613 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:44.613 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:44.613 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:44.613 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:44.620+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:44.623 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:07:44.629 INFO:teuthology.orchestra.run.smithi049.stdout:flattening image image_client.0-clone3 2024-04-24T18:07:44.700 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:44.708+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:44.736 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:07:44.772 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:44.772 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:20 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.356010986000001, 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001123017, 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:44.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422414+0000", 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000112488 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422424+0000", 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0118e-05 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422431+0000", 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9319999999999999e-06 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422477+0000", 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5738999999999997e-05 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422501+0000", 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4032e-05 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.423397+0000", 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089551500000000005 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.423425+0000", 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8775000000000001e-05 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.345691510000002, 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001094629, 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.777 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432621+0000", 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432627+0000", 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5160000000000002e-06 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432635+0000", 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3669999999999997e-06 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432639+0000", 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9179999999999996e-06 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432678+0000", 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8627999999999997e-05 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432701+0000", 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3246000000000001e-05 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433677+0000", 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097568000000000004 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433716+0000", 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9765999999999998e-05 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.345140575999999, 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00093691599999999999, 2024-04-24T18:07:44.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433172+0000", 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433240+0000", 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7769999999999997e-05 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433253+0000", 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3118999999999999e-05 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433259+0000", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3090000000000001e-06 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433287+0000", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7813e-05 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433300+0000", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3004e-05 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.434087+0000", 2024-04-24T18:07:44.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00078750999999999995 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.434110+0000", 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2198999999999999e-05 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:54 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.329581417, 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014283620000000001, 2024-04-24T18:07:44.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448731+0000", 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448738+0000", 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4590000000000003e-06 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448751+0000", 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2833000000000001e-05 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448757+0000", 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448800+0000", 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3282999999999999e-05 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448823+0000", 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2731999999999999e-05 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.450119+0000", 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001295942 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.450160+0000", 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1709999999999999e-05 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.789 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:57 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.326312651999999, 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00095936700000000003, 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452000+0000", 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452057+0000", 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7161999999999999e-05 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452067+0000", 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0278999999999999e-05 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452072+0000", 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9429999999999999e-06 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452106+0000", 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4023000000000003e-05 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452128+0000", 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1145000000000002e-05 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452929+0000", 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00080152800000000003 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452960+0000", 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0939000000000003e-05 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.793 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.319055123999998, 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0009994750000000001, 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459262+0000", 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1609999999999996e-06 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459270+0000", 2024-04-24T18:07:44.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5989999999999996e-06 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459274+0000", 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8539999999999996e-06 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459306+0000", 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1129e-05 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459317+0000", 2024-04-24T18:07:44.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1785e-05 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.460227+0000", 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090945999999999996 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.460258+0000", 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0899999999999999e-05 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.797 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:71 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'45, mlcod=16'43)", 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.316821448999999, 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00093247299999999996, 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461491+0000", 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461555+0000", 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3937999999999997e-05 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461565+0000", 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4800000000000007e-06 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461569+0000", 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5340000000000001e-06 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461616+0000", 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6587e-05 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461640+0000", 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4130000000000001e-05 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462380+0000", 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074047800000000003 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462424+0000", 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3980000000000002e-05 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:72 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'46, mlcod=16'43)", 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.316363386999999, 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010010850000000001, 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461949+0000", 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462032+0000", 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2152999999999998e-05 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462044+0000", 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2170999999999999e-05 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462051+0000", 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9600000000000003e-06 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462081+0000", 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9788e-05 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462094+0000", 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3910999999999999e-05 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462904+0000", 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081001000000000001 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462951+0000", 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6597000000000001e-05 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:83 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.308932463000001, 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001099135, 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469380+0000", 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469387+0000", 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5220000000000004e-06 2024-04-24T18:07:44.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469398+0000", 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1076e-05 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469405+0000", 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4699999999999996e-06 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469441+0000", 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5429999999999998e-05 2024-04-24T18:07:44.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469461+0000", 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9828999999999999e-05 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470443+0000", 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098290399999999994 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470480+0000", 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6544000000000001e-05 2024-04-24T18:07:44.809 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:84 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.308397024000001, 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00094213999999999995, 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:44.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469980+0000", 2024-04-24T18:07:44.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4336000000000002e-05 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469991+0000", 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1191e-05 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470001+0000", 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7680000000000003e-06 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470036+0000", 2024-04-24T18:07:44.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5265000000000002e-05 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470056+0000", 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9646000000000001e-05 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470828+0000", 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077209399999999997 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470858+0000", 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0639000000000003e-05 2024-04-24T18:07:44.813 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:86 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.307230029999999, 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010291390000000001, 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471089+0000", 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3430000000000001e-06 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471104+0000", 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4617000000000001e-05 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471108+0000", 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6430000000000003e-06 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471168+0000", 2024-04-24T18:07:44.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9917000000000002e-05 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471187+0000", 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9147000000000002e-05 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.472073+0000", 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00088590000000000001 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.472112+0000", 2024-04-24T18:07:44.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999999e-05 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:89 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'59, mlcod=16'57)", 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.304438219000001, 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001052597, 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473874+0000", 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473941+0000", 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6638999999999997e-05 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473950+0000", 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.234e-06 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473955+0000", 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5179999999999998e-06 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473991+0000", 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5689000000000003e-05 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474013+0000", 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.209e-05 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474889+0000", 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00087645799999999999 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474928+0000", 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8591999999999999e-05 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:104 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.162092028, 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097261199999999998, 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616220+0000", 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616228+0000", 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1489999999999997e-06 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616239+0000", 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1029000000000001e-05 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616244+0000", 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0340000000000004e-06 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616274+0000", 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9957000000000001e-05 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616290+0000", 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6035000000000001e-05 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617130+0000", 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084017 2024-04-24T18:07:44.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617194+0000", 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4188000000000003e-05 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:105 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.161552743000001, 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001106934, 2024-04-24T18:07:44.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616760+0000", 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616765+0000", 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2839999999999997e-06 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616775+0000", 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7229999999999999e-06 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616780+0000", 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7620000000000003e-06 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616818+0000", 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8637999999999998e-05 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616839+0000", 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0279e-05 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617845+0000", 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010063089999999999 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617868+0000", 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.249e-05 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:44.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.160969659999999, 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010242809999999999, 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617343+0000", 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617408+0000", 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4926999999999997e-05 2024-04-24T18:07:44.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617420+0000", 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1977999999999999e-05 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617423+0000", 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.507e-06 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617458+0000", 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4999999999999997e-05 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617477+0000", 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8902999999999998e-05 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.618336+0000", 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000859044 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.618368+0000", 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1656999999999999e-05 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:109 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:44.835 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.159027469000002, 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099814200000000004, 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619285+0000", 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619338+0000", 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2568999999999998e-05 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619347+0000", 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2350000000000005e-06 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619355+0000", 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8199999999999997e-06 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619377+0000", 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2169999999999999e-05 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619388+0000", 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0569e-05 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620251+0000", 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000863162 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620284+0000", 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3145999999999997e-05 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.839 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'68)", 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.157429755999999, 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097267399999999998, 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620883+0000", 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620950+0000", 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7145999999999997e-05 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620958+0000", 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6240000000000002e-06 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620961+0000", 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3790000000000002e-06 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620992+0000", 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0784999999999997e-05 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621011+0000", 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8811999999999999e-05 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621819+0000", 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00080833000000000001 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621856+0000", 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7004e-05 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.843 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 24.150090916, 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098543200000000006, 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628279+0000", 2024-04-24T18:07:44.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6913000000000002e-05 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628289+0000", 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8539999999999992e-06 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628293+0000", 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4800000000000003e-06 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628329+0000", 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5945000000000002e-05 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628349+0000", 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0367999999999999e-05 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.629181+0000", 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00083184899999999996 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.629208+0000", 2024-04-24T18:07:44.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6676e-05 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:154 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.229518853999998, 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001000143, 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548799+0000", 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9579999999999998e-06 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548808+0000", 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7220000000000005e-06 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548813+0000", 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0690000000000001e-06 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548846+0000", 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3419e-05 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548857+0000", 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.094e-05 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549758+0000", 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090098800000000003 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549795+0000", 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6531e-05 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:155 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.22912187, 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096950400000000005, 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549275+0000", 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4623999999999995e-05 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549283+0000", 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590000000000002e-06 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549292+0000", 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3340000000000008e-06 2024-04-24T18:07:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549331+0000", 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8844999999999998e-05 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549351+0000", 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9791000000000001e-05 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.550128+0000", 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077682700000000001 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.550161+0000", 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3318999999999997e-05 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:44.856 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:44.857 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #2, image image_client.0-clone2 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent3 2024-04-24T18:07:44.858 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:07:44.859 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:44.866+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:44.913 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:44.933 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:44.952 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:44.958 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:44.966+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:44.971 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:44.990 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:45.010 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:45.030 INFO:teuthology.orchestra.run.smithi049.stdout:341 write 0xbf86a16 thru 0xbf8f949 (0x8f34 bytes) 2024-04-24T18:07:45.030 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:45.036 INFO:teuthology.orchestra.run.smithi049.stdout:342 read 0x6624cba thru 0x6631f21 (0xd268 bytes) 2024-04-24T18:07:45.036 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:45.036 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:45.046 INFO:teuthology.orchestra.run.smithi049.stdout:344 trunc from 0xd345ad0 to 0x3f151b4 2024-04-24T18:07:45.049 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:45.051 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:45.052 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:45.052 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:45.052 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:45.052 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:45.055 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:07:45.065 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:45.073+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:45.069 INFO:teuthology.orchestra.run.smithi049.stdout:345 read 0x165c0e8 thru 0x1664343 (0x825c bytes) 2024-04-24T18:07:45.078 INFO:teuthology.orchestra.run.smithi049.stdout:346 writesame 0x724f89f thru 0x7250eac data_size 0x3(0x160e bytes) 2024-04-24T18:07:45.082 INFO:teuthology.orchestra.run.smithi049.stdout:347 trunc from 0x7250ead to 0xa374dae 2024-04-24T18:07:45.088 INFO:teuthology.orchestra.run.smithi049.stdout:348 read 0x59d7155 thru 0x59dfa5f (0x890b bytes) 2024-04-24T18:07:45.088 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.088 INFO:teuthology.orchestra.run.smithi049.stdout:350 writesame 0x92a11f3 thru 0x92a2670 data_size 0x2b(0x147e bytes) 2024-04-24T18:07:45.091 INFO:teuthology.orchestra.run.smithi049.stdout:351 write 0x45f0c49 thru 0x45f5e90 (0x5248 bytes) 2024-04-24T18:07:45.094 INFO:teuthology.orchestra.run.smithi049.stdout:352 read 0x61d8bed thru 0x61daac3 (0x1ed7 bytes) 2024-04-24T18:07:45.095 INFO:teuthology.orchestra.run.smithi049.stdout:355 trunc from 0xa374dae to 0xae334f9 2024-04-24T18:07:45.101 INFO:teuthology.orchestra.run.smithi049.stdout:356 read 0x714f180 thru 0x71522d6 (0x3157 bytes) 2024-04-24T18:07:45.101 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.101 INFO:teuthology.orchestra.run.smithi049.stdout:358 writesame 0x780ec00 thru 0x781de34 data_size 0x1(0xf235 bytes) 2024-04-24T18:07:45.133 INFO:teuthology.orchestra.run.smithi049.stdout:359 flatten 2024-04-24T18:07:45.147 INFO:teuthology.orchestra.run.smithi049.stdout:360 read 0x96e8d61 thru 0x96f1fd4 (0x9274 bytes) 2024-04-24T18:07:45.150 INFO:teuthology.orchestra.run.smithi049.stdout:362 trunc from 0xae334f9 to 0x88a971a 2024-04-24T18:07:45.156 INFO:teuthology.orchestra.run.smithi049.stdout:363 writesame 0x2a4c556 thru 0x2a56add data_size 0x4(0xa588 bytes) 2024-04-24T18:07:45.160 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:45.168+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:45.173 INFO:teuthology.orchestra.run.smithi049.stdout:366 trunc from 0x88a971a to 0x1adc6e 2024-04-24T18:07:45.196 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.202 INFO:teuthology.orchestra.run.smithi049.stdout:368 write 0x1059167 thru 0x1062b6a (0x9a04 bytes) 2024-04-24T18:07:45.215 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:45.216 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:45.224 INFO:teuthology.orchestra.run.smithi049.stdout:369 write 0xd36df71 thru 0xd3778cf (0x995f bytes) 2024-04-24T18:07:45.227 INFO:teuthology.orchestra.run.smithi049.stdout:372 write 0xc6d3e50 thru 0xc6ddac3 (0x9c74 bytes) 2024-04-24T18:07:45.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:07:45.230 INFO:teuthology.orchestra.run.smithi049.stdout:373 writesame 0x519a33b thru 0x51a5479 data_size 0x4b(0xb13f bytes) 2024-04-24T18:07:45.235 INFO:teuthology.orchestra.run.smithi049.stdout:374 read 0x8df785f thru 0x8dfc6d0 (0x4e72 bytes) 2024-04-24T18:07:45.235 INFO:teuthology.orchestra.run.smithi049.stdout:375 read 0x5f5d92e thru 0x5f6ce02 (0xf4d5 bytes) 2024-04-24T18:07:45.235 INFO:teuthology.orchestra.run.smithi049.stdout:376 writesame 0x4b15bcb thru 0x4b23c07 data_size 0x81(0xe03d bytes) 2024-04-24T18:07:45.236 INFO:teuthology.orchestra.run.smithi049.stdout:377 read 0xb4f336 thru 0xb56cea (0x79b5 bytes) 2024-04-24T18:07:45.237 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.237 INFO:teuthology.orchestra.run.smithi049.stdout:379 punch from 0x554845a to 0x554ae70, (0x2a16 bytes) 2024-04-24T18:07:45.243 INFO:teuthology.orchestra.run.smithi049.stdout:380 write 0xe8c9a27 thru 0xe8d82bf (0xe899 bytes) 2024-04-24T18:07:45.247 INFO:teuthology.orchestra.run.smithi049.stdout:381 read 0x83839a6 thru 0x838ada5 (0x7400 bytes) 2024-04-24T18:07:45.247 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.247 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.247 INFO:teuthology.orchestra.run.smithi049.stdout:384 read 0x8ed7b7 thru 0x8f1288 (0x3ad2 bytes) 2024-04-24T18:07:45.247 INFO:teuthology.orchestra.run.smithi049.stdout:385 write 0x372b7a4 thru 0x3738079 (0xc8d6 bytes) 2024-04-24T18:07:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:386 read 0xa4aee77 thru 0xa4af68c (0x816 bytes) 2024-04-24T18:07:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:387 punch from 0x4811dbb to 0x481bc1d, (0x9e62 bytes) 2024-04-24T18:07:45.250 INFO:teuthology.orchestra.run.smithi049.stdout:389 read 0xb8ff9f7 thru 0xb90d394 (0xd99e bytes) 2024-04-24T18:07:45.259 INFO:teuthology.orchestra.run.smithi049.stdout:391 trunc from 0xe8d82c0 to 0x6d01471 2024-04-24T18:07:45.262 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:45.270+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:45.276 INFO:teuthology.orchestra.run.smithi049.stdout:392 trunc from 0x6d01471 to 0xc08597 2024-04-24T18:07:45.291 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.292 INFO:teuthology.orchestra.run.smithi049.stdout:396 trunc from 0xc08597 to 0x1c5cb99 2024-04-24T18:07:45.297 INFO:teuthology.orchestra.run.smithi049.stdout:397 read 0x92c011 thru 0x92caba (0xaaa bytes) 2024-04-24T18:07:45.297 INFO:teuthology.orchestra.run.smithi049.stdout:398 read 0xbb85bc thru 0xbc2c8f (0xa6d4 bytes) 2024-04-24T18:07:45.307 INFO:teuthology.orchestra.run.smithi049.stdout:399 trunc from 0x1c5cb99 to 0xac3d10b 2024-04-24T18:07:45.312 INFO:teuthology.orchestra.run.smithi049.stdout:400 punch from 0x5be90f4 to 0x5bebad4, (0x29e0 bytes) 2024-04-24T18:07:45.313 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.317 INFO:teuthology.orchestra.run.smithi049.stdout:406 trunc from 0xac3d10b to 0x66578cb 2024-04-24T18:07:45.322 INFO:teuthology.orchestra.run.smithi049.stdout:409 writesame 0x22f882a thru 0x22fccff data_size 0x21(0x44d6 bytes) 2024-04-24T18:07:45.326 INFO:teuthology.orchestra.run.smithi049.stdout:410 read 0x7ab8de thru 0x7ac96a (0x108d bytes) 2024-04-24T18:07:45.337 INFO:teuthology.orchestra.run.smithi049.stdout:411 write 0xb58509d thru 0xb589e31 (0x4d95 bytes) 2024-04-24T18:07:45.340 INFO:teuthology.orchestra.run.smithi049.stdout:412 write 0x5db9779 thru 0x5dbfad4 (0x635c bytes) 2024-04-24T18:07:45.355 INFO:teuthology.orchestra.run.smithi049.stdout:413 trunc from 0xb589e32 to 0x1053ff 2024-04-24T18:07:45.362 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:45.371+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:45.373 INFO:teuthology.orchestra.run.smithi049.stdout:414 trunc from 0x1053ff to 0x458a492 2024-04-24T18:07:45.378 INFO:teuthology.orchestra.run.smithi049.stdout:415 read 0xe92aa9 thru 0xea0324 (0xd87c bytes) 2024-04-24T18:07:45.380 INFO:teuthology.orchestra.run.smithi049.stdout:416 trunc from 0x458a492 to 0x618ddc5 2024-04-24T18:07:45.383 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:45.384 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:45.386 INFO:teuthology.orchestra.run.smithi049.stdout:417 read 0x4472e13 thru 0x4474000 (0x11ee bytes) 2024-04-24T18:07:45.386 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.386 INFO:teuthology.orchestra.run.smithi049.stdout:421 read 0x3b8b47d thru 0x3b8d38a (0x1f0e bytes) 2024-04-24T18:07:45.386 INFO:teuthology.orchestra.run.smithi049.stdout:422 read 0x3801691 thru 0x380e35d (0xcccd bytes) 2024-04-24T18:07:45.391 INFO:teuthology.orchestra.run.smithi049.stdout:423 trunc from 0x618ddc5 to 0x1804dfe 2024-04-24T18:07:45.399 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:07:45.403 INFO:teuthology.orchestra.run.smithi049.stdout:425 write 0x27f9775 thru 0x2802f06 (0x9792 bytes) 2024-04-24T18:07:45.412 INFO:teuthology.orchestra.run.smithi049.stdout:426 writesame 0x2dde17c thru 0x2de9306 data_size 0x2b(0xb18b bytes) 2024-04-24T18:07:45.412 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.412 INFO:teuthology.orchestra.run.smithi049.stdout:431 punch from 0x230741d to 0x230fc9d, (0x8880 bytes) 2024-04-24T18:07:45.412 INFO:teuthology.orchestra.run.smithi049.stdout:432 read 0x177aa09 thru 0x177e570 (0x3b68 bytes) 2024-04-24T18:07:45.412 INFO:teuthology.orchestra.run.smithi049.stdout:433 read 0x2b787a4 thru 0x2b83fc3 (0xb820 bytes) 2024-04-24T18:07:45.429 INFO:teuthology.orchestra.run.smithi049.stdout:434 writesame 0xcfe4b00 thru 0xcfebb3b data_size 0xb(0x703c bytes) 2024-04-24T18:07:45.429 INFO:teuthology.orchestra.run.smithi049.stdout:435 write 0x7e007e6 thru 0x7e0b7c5 (0xafe0 bytes) 2024-04-24T18:07:45.433 INFO:teuthology.orchestra.run.smithi049.stdout:436 write 0x63bcd21 thru 0x63c119f (0x447f bytes) 2024-04-24T18:07:45.437 INFO:teuthology.orchestra.run.smithi049.stdout:437 write 0x4aa1941 thru 0x4aaa50f (0x8bcf bytes) 2024-04-24T18:07:45.439 INFO:teuthology.orchestra.run.smithi049.stdout:438 write 0x23064e1 thru 0x230cfda (0x6afa bytes) 2024-04-24T18:07:45.442 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.442 INFO:teuthology.orchestra.run.smithi049.stdout:442 punch from 0xc38dd0f to 0xc38df82, (0x273 bytes) 2024-04-24T18:07:45.443 INFO:teuthology.orchestra.run.smithi049.stdout:443 writesame 0x91b25bf thru 0x91b950e data_size 0x68(0x6f50 bytes) 2024-04-24T18:07:45.446 INFO:teuthology.orchestra.run.smithi049.stdout:444 read 0x8ea7756 thru 0x8eb1373 (0x9c1e bytes) 2024-04-24T18:07:45.446 INFO:teuthology.orchestra.run.smithi049.stdout:445 read 0x4a03c9f thru 0x4a0bc0f (0x7f71 bytes) 2024-04-24T18:07:45.446 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.446 INFO:teuthology.orchestra.run.smithi049.stdout:448 read 0x491a2f4 thru 0x491a5c1 (0x2ce bytes) 2024-04-24T18:07:45.446 INFO:teuthology.orchestra.run.smithi049.stdout:449 write 0x6a2d568 thru 0x6a31686 (0x411f bytes) 2024-04-24T18:07:45.449 INFO:teuthology.orchestra.run.smithi049.stdout:450 write 0x729c6cf thru 0x72a60c0 (0x99f2 bytes) 2024-04-24T18:07:45.452 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.452 INFO:teuthology.orchestra.run.smithi049.stdout:454 read 0x12d93a1 thru 0x12e6c4e (0xd8ae bytes) 2024-04-24T18:07:45.452 INFO:teuthology.orchestra.run.smithi049.stdout:455 write 0xcf560e4 thru 0xcf5e0ec (0x8009 bytes) 2024-04-24T18:07:45.455 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.455 INFO:teuthology.orchestra.run.smithi049.stdout:457 writesame 0xcdf521d thru 0xcdfc142 data_size 0x2(0x6f26 bytes) 2024-04-24T18:07:45.463 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:45.471+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:45.464 INFO:teuthology.orchestra.run.smithi049.stdout:458 write 0x4e68653 thru 0x4e6f078 (0x6a26 bytes) 2024-04-24T18:07:45.465 INFO:teuthology.orchestra.run.smithi049.stdout:459 write 0x42183b4 thru 0x421b073 (0x2cc0 bytes) 2024-04-24T18:07:45.468 INFO:teuthology.orchestra.run.smithi049.stdout:460 read 0xcb2954 thru 0xcbc600 (0x9cad bytes) 2024-04-24T18:07:45.475 INFO:teuthology.orchestra.run.smithi049.stdout:462 writesame 0xdb8b54b thru 0xdb96e6a data_size 0x10(0xb920 bytes) 2024-04-24T18:07:45.475 INFO:teuthology.orchestra.run.smithi049.stdout:463 write 0xa3f5085 thru 0xa404acc (0xfa48 bytes) 2024-04-24T18:07:45.478 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.479 INFO:teuthology.orchestra.run.smithi049.stdout:465 read 0x8e086fc thru 0x8e0b89a (0x319f bytes) 2024-04-24T18:07:45.479 INFO:teuthology.orchestra.run.smithi049.stdout:466 read 0x8a419aa thru 0x8a4a34e (0x89a5 bytes) 2024-04-24T18:07:45.479 INFO:teuthology.orchestra.run.smithi049.stdout:467 write 0xcd843e3 thru 0xcd9222c (0xde4a bytes) 2024-04-24T18:07:45.480 INFO:teuthology.orchestra.run.smithi049.stdout:469 punch from 0x6ea23b to 0x6f7938, (0xd6fd bytes) 2024-04-24T18:07:45.480 INFO:teuthology.orchestra.run.smithi049.stdout:470 write 0xaa68d45 thru 0xaa722a5 (0x9561 bytes) 2024-04-24T18:07:45.485 INFO:teuthology.orchestra.run.smithi049.stdout:471 trunc from 0xdb96e6b to 0xc6c7ce5 2024-04-24T18:07:45.492 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.492 INFO:teuthology.orchestra.run.smithi049.stdout:473 punch from 0x1069f4a to 0x10756f2, (0xb7a8 bytes) 2024-04-24T18:07:45.492 INFO:teuthology.orchestra.run.smithi049.stdout:476 punch from 0x36e0564 to 0x36ecaeb, (0xc587 bytes) 2024-04-24T18:07:45.492 INFO:teuthology.orchestra.run.smithi049.stdout:477 read 0xa7d5883 thru 0xa7e398e (0xe10c bytes) 2024-04-24T18:07:45.492 INFO:teuthology.orchestra.run.smithi049.stdout:478 punch from 0x97bd8b2 to 0x97c12fe, (0x3a4c bytes) 2024-04-24T18:07:45.492 INFO:teuthology.orchestra.run.smithi049.stdout:479 writesame 0x246621d thru 0x2468f52 data_size 0x9(0x2d36 bytes) 2024-04-24T18:07:45.494 INFO:teuthology.orchestra.run.smithi049.stdout:484 read 0xc6090e5 thru 0xc60f572 (0x648e bytes) 2024-04-24T18:07:45.495 INFO:teuthology.orchestra.run.smithi049.stdout:485 read 0x761775c thru 0x7626d2b (0xf5d0 bytes) 2024-04-24T18:07:45.495 INFO:teuthology.orchestra.run.smithi049.stdout:486 writesame 0x87130bb thru 0x87169da data_size 0x10(0x3920 bytes) 2024-04-24T18:07:45.496 INFO:teuthology.orchestra.run.smithi049.stdout:487 write 0x8c3f412 thru 0x8c41bf4 (0x27e3 bytes) 2024-04-24T18:07:45.499 INFO:teuthology.orchestra.run.smithi049.stdout:489 read 0x29faa8c thru 0x29fdc47 (0x31bc bytes) 2024-04-24T18:07:45.499 INFO:teuthology.orchestra.run.smithi049.stdout:491 writesame 0x509812f thru 0x509fa9a data_size 0x26(0x796c bytes) 2024-04-24T18:07:45.502 INFO:teuthology.orchestra.run.smithi049.stdout:492 read 0x2702950 thru 0x2709323 (0x69d4 bytes) 2024-04-24T18:07:45.502 INFO:teuthology.orchestra.run.smithi049.stdout:493 read 0x3b80054 thru 0x3b8a426 (0xa3d3 bytes) 2024-04-24T18:07:45.502 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.503 INFO:teuthology.orchestra.run.smithi049.stdout:495 punch from 0x27d279a to 0x27e2173, (0xf9d9 bytes) 2024-04-24T18:07:45.503 INFO:teuthology.orchestra.run.smithi049.stdout:496 read 0xb99588 thru 0xba4668 (0xb0e1 bytes) 2024-04-24T18:07:45.503 INFO:teuthology.orchestra.run.smithi049.stdout:498 read 0x5fc6f thru 0x6c78c (0xcb1e bytes) 2024-04-24T18:07:45.508 INFO:teuthology.orchestra.run.smithi049.stdout:499 trunc from 0xc6c7ce5 to 0x7bceae9 2024-04-24T18:07:45.532 INFO:teuthology.orchestra.run.smithi049.stdout:502 trunc from 0x7bceae9 to 0xd17afb9 2024-04-24T18:07:45.537 INFO:teuthology.orchestra.run.smithi049.stdout:503 writesame 0x8c31fb7 thru 0x8c40a53 data_size 0x1(0xea9d bytes) 2024-04-24T18:07:45.554 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:45.554 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:45.564 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:45.573+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:45.567 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:07:45.567 INFO:teuthology.orchestra.run.smithi049.stdout:504 read 0x938aaa9 thru 0x9394b93 (0xa0eb bytes) 2024-04-24T18:07:45.567 INFO:teuthology.orchestra.run.smithi049.stdout:505 write 0xcbe0f4e thru 0xcbe61fb (0x52ae bytes) 2024-04-24T18:07:45.570 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:07:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:508 punch from 0xa9e45af to 0xa9e7915, (0x3366 bytes) 2024-04-24T18:07:45.573 INFO:teuthology.orchestra.run.smithi049.stdout:511 write 0x3dd01dd thru 0x3dd3938 (0x375c bytes) 2024-04-24T18:07:45.580 INFO:teuthology.orchestra.run.smithi049.stdout:512 trunc from 0xd17afb9 to 0x97c0756 2024-04-24T18:07:45.603 INFO:teuthology.orchestra.run.smithi049.stdout:513 write 0xe663833 thru 0xe666d0b (0x34d9 bytes) 2024-04-24T18:07:45.606 INFO:teuthology.orchestra.run.smithi049.stdout:514 writesame 0x8852367 thru 0x885b643 data_size 0x7(0x92dd bytes) 2024-04-24T18:07:45.610 INFO:teuthology.orchestra.run.smithi049.stdout:515 read 0x69c07a5 thru 0x69c3625 (0x2e81 bytes) 2024-04-24T18:07:45.611 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.611 INFO:teuthology.orchestra.run.smithi049.stdout:517 write 0xc815db8 thru 0xc8254cb (0xf714 bytes) 2024-04-24T18:07:45.613 INFO:teuthology.orchestra.run.smithi049.stdout:518 punch from 0x98c4f9a to 0x98cfcf0, (0xad56 bytes) 2024-04-24T18:07:45.614 INFO:teuthology.orchestra.run.smithi049.stdout:519 read 0x27fba8 thru 0x280879 (0xcd2 bytes) 2024-04-24T18:07:45.614 INFO:teuthology.orchestra.run.smithi049.stdout:521 read 0xe1ec69f thru 0xe1f43cf (0x7d31 bytes) 2024-04-24T18:07:45.614 INFO:teuthology.orchestra.run.smithi049.stdout:526 writesame 0x561a864 thru 0x5626598 data_size 0x1(0xbd35 bytes) 2024-04-24T18:07:45.637 INFO:teuthology.orchestra.run.smithi049.stdout:527 write 0x1b52231 thru 0x1b5e557 (0xc327 bytes) 2024-04-24T18:07:45.640 INFO:teuthology.orchestra.run.smithi049.stdout:528 read 0xad92898 thru 0xad9b1f8 (0x8961 bytes) 2024-04-24T18:07:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:529 punch from 0x7a1138c to 0x7a1e858, (0xd4cc bytes) 2024-04-24T18:07:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:530 punch from 0x51bac10 to 0x51c2b9f, (0x7f8f bytes) 2024-04-24T18:07:45.641 INFO:teuthology.orchestra.run.smithi049.stdout:531 read 0x279464 thru 0x27eb65 (0x5702 bytes) 2024-04-24T18:07:45.655 INFO:teuthology.orchestra.run.smithi049.stdout:532 trunc from 0xe666d0c to 0x228a7f5 2024-04-24T18:07:45.665 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:45.673+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:45.673 INFO:teuthology.orchestra.run.smithi049.stdout:533 read 0x17176dc thru 0x171fb8b (0x84b0 bytes) 2024-04-24T18:07:45.681 INFO:teuthology.orchestra.run.smithi049.stdout:534 write 0x4ba3049 thru 0x4baaa82 (0x7a3a bytes) 2024-04-24T18:07:45.684 INFO:teuthology.orchestra.run.smithi049.stdout:535 read 0x493cfe2 thru 0x4942b6a (0x5b89 bytes) 2024-04-24T18:07:45.694 INFO:teuthology.orchestra.run.smithi049.stdout:536 write 0x95e9ee2 thru 0x95efd5c (0x5e7b bytes) 2024-04-24T18:07:45.701 INFO:teuthology.orchestra.run.smithi049.stdout:537 trunc from 0x95efd5d to 0x5c8963f 2024-04-24T18:07:45.712 INFO:teuthology.orchestra.run.smithi049.stdout:539 writesame 0x541e3fc thru 0x5420b62 data_size 0xb(0x2767 bytes) 2024-04-24T18:07:45.716 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:45.716 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:45.723 INFO:teuthology.orchestra.run.smithi049.stdout:540 writesame 0x7fe5396 thru 0x7ff4561 data_size 0x32(0xf1cc bytes) 2024-04-24T18:07:45.723 INFO:teuthology.orchestra.run.smithi049.stdout:541 writesame 0x5c59e68 thru 0x5c65055 data_size 0x2(0xb1ee bytes) 2024-04-24T18:07:45.728 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:07:45.748 INFO:teuthology.orchestra.run.smithi049.stdout:542 writesame 0xe169bba thru 0xe1765e9 data_size 0x10(0xca30 bytes) 2024-04-24T18:07:45.748 INFO:teuthology.orchestra.run.smithi049.stdout:543 write 0x49eb367 thru 0x49f6d69 (0xba03 bytes) 2024-04-24T18:07:45.751 INFO:teuthology.orchestra.run.smithi049.stdout:544 writesame 0x373e721 thru 0x3747a62 data_size 0x7a(0x9342 bytes) 2024-04-24T18:07:45.753 INFO:teuthology.orchestra.run.smithi049.stdout:546 writesame 0x3ec6e91 thru 0x3ecaf87 data_size 0x1(0x40f7 bytes) 2024-04-24T18:07:45.762 INFO:teuthology.orchestra.run.smithi049.stdout:549 read 0xcf9facb thru 0xcfa1cf4 (0x222a bytes) 2024-04-24T18:07:45.762 INFO:teuthology.orchestra.run.smithi049.stdout:550 punch from 0x8200ba3 to 0x820577c, (0x4bd9 bytes) 2024-04-24T18:07:45.763 INFO:teuthology.orchestra.run.smithi049.stdout:truncating to largest ever: 0xe9dd9b9 2024-04-24T18:07:45.763 INFO:teuthology.orchestra.run.smithi049.stdout:551 trunc from 0xe1765ea to 0xe9dd9b9 2024-04-24T18:07:45.766 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:45.774+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:45.769 INFO:teuthology.orchestra.run.smithi049.stdout:552 write 0xdf889ca thru 0xdf8a2f3 (0x192a bytes) 2024-04-24T18:07:45.772 INFO:teuthology.orchestra.run.smithi049.stdout:553 read 0xa3c5e47 thru 0xa3d4ba9 (0xed63 bytes) 2024-04-24T18:07:45.772 INFO:teuthology.orchestra.run.smithi049.stdout:554 write 0x1f1bea3 thru 0x1f21bb5 (0x5d13 bytes) 2024-04-24T18:07:45.775 INFO:teuthology.orchestra.run.smithi049.stdout:555 write 0x22d606a thru 0x22d7f05 (0x1e9c bytes) 2024-04-24T18:07:45.776 INFO:teuthology.orchestra.run.smithi049.stdout:558 write 0xcdc6016 thru 0xcdc7e6d (0x1e58 bytes) 2024-04-24T18:07:45.779 INFO:teuthology.orchestra.run.smithi049.stdout:560 read 0xc752ccb thru 0xc75a4d6 (0x780c bytes) 2024-04-24T18:07:45.784 INFO:teuthology.orchestra.run.smithi049.stdout:561 trunc from 0xe9dd9b9 to 0xae82318 2024-04-24T18:07:45.794 INFO:teuthology.orchestra.run.smithi049.stdout:564 read 0x7a86801 thru 0x7a8e542 (0x7d42 bytes) 2024-04-24T18:07:45.794 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.794 INFO:teuthology.orchestra.run.smithi049.stdout:569 read 0xac99371 thru 0xaca57df (0xc46f bytes) 2024-04-24T18:07:45.794 INFO:teuthology.orchestra.run.smithi049.stdout:570 writesame 0x7d2e692 thru 0x7d311c5 data_size 0x23(0x2b34 bytes) 2024-04-24T18:07:45.798 INFO:teuthology.orchestra.run.smithi049.stdout:571 trunc from 0xae82318 to 0xb6df47f 2024-04-24T18:07:45.810 INFO:teuthology.orchestra.run.smithi049.stdout:572 write 0xd1fbdc4 thru 0xd20a5a8 (0xe7e5 bytes) 2024-04-24T18:07:45.815 INFO:teuthology.orchestra.run.smithi049.stdout:573 trunc from 0xd20a5a9 to 0xcc0bc3a 2024-04-24T18:07:45.821 INFO:teuthology.orchestra.run.smithi049.stdout:574 writesame 0xae3b30b thru 0xae4258f data_size 0x1(0x7285 bytes) 2024-04-24T18:07:45.836 INFO:teuthology.orchestra.run.smithi049.stdout:575 read 0x935c6b5 thru 0x935e85c (0x21a8 bytes) 2024-04-24T18:07:45.842 INFO:teuthology.orchestra.run.smithi049.stdout:576 write 0xd502573 thru 0xd50fd1c (0xd7aa bytes) 2024-04-24T18:07:45.846 INFO:teuthology.orchestra.run.smithi049.stdout:577 write 0x81121a0 thru 0x812073f (0xe5a0 bytes) 2024-04-24T18:07:45.848 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.848 INFO:teuthology.orchestra.run.smithi049.stdout:579 write 0x6f19ac5 thru 0x6f25868 (0xbda4 bytes) 2024-04-24T18:07:45.851 INFO:teuthology.orchestra.run.smithi049.stdout:581 read 0x8162d42 thru 0x816a469 (0x7728 bytes) 2024-04-24T18:07:45.851 INFO:teuthology.orchestra.run.smithi049.stdout:583 write 0x9c3b6c0 thru 0x9c40f8b (0x58cc bytes) 2024-04-24T18:07:45.853 INFO:teuthology.orchestra.run.smithi049.stdout:584 writesame 0x4975750 thru 0x4981789 data_size 0x46(0xc03a bytes) 2024-04-24T18:07:45.854 INFO:teuthology.orchestra.run.smithi049.stdout:587 read 0xcd1b770 thru 0xcd299d0 (0xe261 bytes) 2024-04-24T18:07:45.854 INFO:teuthology.orchestra.run.smithi049.stdout:588 writesame 0x75711d9 thru 0x7574cfb data_size 0x1(0x3b23 bytes) 2024-04-24T18:07:45.867 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:45.875+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:45.876 INFO:teuthology.orchestra.run.smithi049.stdout:589 trunc from 0xd50fd1d to 0x22a2a36 2024-04-24T18:07:45.897 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:45.897 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:45.905 INFO:teuthology.orchestra.run.smithi049.stdout:591 write 0xc93b0ab thru 0xc9493dd (0xe333 bytes) 2024-04-24T18:07:45.910 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:07:45.920 INFO:teuthology.orchestra.run.smithi049.stdout:592 trunc from 0xc9493de to 0x1a109d0 2024-04-24T18:07:45.938 INFO:teuthology.orchestra.run.smithi049.stdout:593 write 0x2897688 thru 0x289eb04 (0x747d bytes) 2024-04-24T18:07:45.959 INFO:teuthology.orchestra.run.smithi049.stdout:595 write 0xde36910 thru 0xde44910 (0xe001 bytes) 2024-04-24T18:07:45.962 INFO:teuthology.orchestra.run.smithi049.stdout:596 writesame 0x982cccf thru 0x983814c data_size 0x97(0xb47e bytes) 2024-04-24T18:07:45.964 INFO:teuthology.orchestra.run.smithi049.stdout:597 read 0x16b6852 thru 0x16bb5aa (0x4d59 bytes) 2024-04-24T18:07:45.965 INFO:teuthology.orchestra.run.smithi049.stdout:598 writesame 0xa8e2acc thru 0xa8ef8f7 data_size 0x46(0xce2c bytes) 2024-04-24T18:07:45.967 INFO:teuthology.orchestra.run.smithi049.stdout:600 read 0x5fa038c thru 0x5fac77c (0xc3f1 bytes) 2024-04-24T18:07:45.968 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:45.968 INFO:teuthology.orchestra.run.smithi049.stdout:604 writesame 0xcf30649 thru 0xcf31159 data_size 0x1(0xb11 bytes) 2024-04-24T18:07:45.968 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:45.976+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:45.977 INFO:teuthology.orchestra.run.smithi049.stdout:605 trunc from 0xde44911 to 0x840b5e2 2024-04-24T18:07:45.988 INFO:teuthology.orchestra.run.smithi049.stdout:607 read 0x75fbe52 thru 0x7602096 (0x6245 bytes) 2024-04-24T18:07:45.995 INFO:teuthology.orchestra.run.smithi049.stdout:608 trunc from 0x840b5e2 to 0xddbc796 2024-04-24T18:07:46.000 INFO:teuthology.orchestra.run.smithi049.stdout:609 read 0x28b7a1e thru 0x28b9569 (0x1b4c bytes) 2024-04-24T18:07:46.001 INFO:teuthology.orchestra.run.smithi049.stdout:610 read 0xf9033e thru 0xf98b72 (0x8835 bytes) 2024-04-24T18:07:46.001 INFO:teuthology.orchestra.run.smithi049.stdout:611 write 0x4a1e1c thru 0x4b17b4 (0xf999 bytes) 2024-04-24T18:07:46.015 INFO:teuthology.orchestra.run.smithi049.stdout:613 trunc from 0xddbc796 to 0x35fffca 2024-04-24T18:07:46.027 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.028 INFO:teuthology.orchestra.run.smithi049.stdout:616 read 0x13a9494 thru 0x13ace91 (0x39fe bytes) 2024-04-24T18:07:46.040 INFO:teuthology.orchestra.run.smithi049.stdout:617 writesame 0x8ee270b thru 0x8ee3769 data_size 0xb(0x105f bytes) 2024-04-24T18:07:46.040 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:46.040 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:46.050 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:07:46.053 INFO:teuthology.orchestra.run.smithi049.stdout:618 write 0xe796ac4 thru 0xe798699 (0x1bd6 bytes) 2024-04-24T18:07:46.059 INFO:teuthology.orchestra.run.smithi049.stdout:619 trunc from 0xe79869a to 0xa9d8d93 2024-04-24T18:07:46.068 INFO:teuthology.orchestra.run.smithi049.stdout:620 read 0x669392e thru 0x6694792 (0xe65 bytes) 2024-04-24T18:07:46.069 INFO:teuthology.orchestra.run.smithi049.stdout:621 read 0xa8fcb50 thru 0xa906978 (0x9e29 bytes) 2024-04-24T18:07:46.070 INFO:teuthology.orchestra.run.smithi049.stdout:622 trunc from 0xa9d8d93 to 0xc9c3e4d 2024-04-24T18:07:46.090 INFO:teuthology.orchestra.run.smithi049.stdout:623 trunc from 0xc9c3e4d to 0x2559b 2024-04-24T18:07:46.092 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:46.100+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:46.108 INFO:teuthology.orchestra.run.smithi049.stdout:624 punch from 0x10763 to 0x12b63, (0x2400 bytes) 2024-04-24T18:07:46.108 INFO:teuthology.orchestra.run.smithi049.stdout:626 read 0x209fe thru 0x2559a (0x4b9d bytes) 2024-04-24T18:07:46.110 INFO:teuthology.orchestra.run.smithi049.stdout:628 trunc from 0x2559b to 0x29c4787 2024-04-24T18:07:46.116 INFO:teuthology.orchestra.run.smithi049.stdout:630 punch from 0x1da22be to 0x1dac340, (0xa082 bytes) 2024-04-24T18:07:46.129 INFO:teuthology.orchestra.run.smithi049.stdout:632 write 0x810f4b0 thru 0x81143cd (0x4f1e bytes) 2024-04-24T18:07:46.131 INFO:teuthology.orchestra.run.smithi049.stdout:633 read 0x30030af thru 0x3008485 (0x53d7 bytes) 2024-04-24T18:07:46.132 INFO:teuthology.orchestra.run.smithi049.stdout:635 read 0x3e7fb98 thru 0x3e803f8 (0x861 bytes) 2024-04-24T18:07:46.132 INFO:teuthology.orchestra.run.smithi049.stdout:636 write 0x27c01eb thru 0x27c58d3 (0x56e9 bytes) 2024-04-24T18:07:46.134 INFO:teuthology.orchestra.run.smithi049.stdout:637 writesame 0x7d5e7c5 thru 0x7d68605 data_size 0x1d(0x9e41 bytes) 2024-04-24T18:07:46.137 INFO:teuthology.orchestra.run.smithi049.stdout:638 write 0x286b09e thru 0x287464b (0x95ae bytes) 2024-04-24T18:07:46.140 INFO:teuthology.orchestra.run.smithi049.stdout:639 read 0x13f5a14 thru 0x13fa724 (0x4d11 bytes) 2024-04-24T18:07:46.140 INFO:teuthology.orchestra.run.smithi049.stdout:640 punch from 0x52cdf1c to 0x52cf3ce, (0x14b2 bytes) 2024-04-24T18:07:46.145 INFO:teuthology.orchestra.run.smithi049.stdout:641 write 0x8534e7e thru 0x8540aec (0xbc6f bytes) 2024-04-24T18:07:46.148 INFO:teuthology.orchestra.run.smithi049.stdout:642 write 0x7a02511 thru 0x7a065a8 (0x4098 bytes) 2024-04-24T18:07:46.150 INFO:teuthology.orchestra.run.smithi049.stdout:643 read 0x79a276f thru 0x79add71 (0xb603 bytes) 2024-04-24T18:07:46.151 INFO:teuthology.orchestra.run.smithi049.stdout:644 punch from 0x5a2f909 to 0x5a38c18, (0x930f bytes) 2024-04-24T18:07:46.153 INFO:teuthology.orchestra.run.smithi049.stdout:647 trunc from 0x8540aed to 0x646d752 2024-04-24T18:07:46.163 INFO:teuthology.orchestra.run.smithi049.stdout:648 write 0x2347d8f thru 0x234fe5f (0x80d1 bytes) 2024-04-24T18:07:46.166 INFO:teuthology.orchestra.run.smithi049.stdout:650 read 0x4ba701f thru 0x4baed08 (0x7cea bytes) 2024-04-24T18:07:46.166 INFO:teuthology.orchestra.run.smithi049.stdout:651 write 0x7b67c6 thru 0x7bcfcf (0x680a bytes) 2024-04-24T18:07:46.167 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.167 INFO:teuthology.orchestra.run.smithi049.stdout:653 write 0x4f92c94 thru 0x4f98564 (0x58d1 bytes) 2024-04-24T18:07:46.169 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:46.177+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:46.169 INFO:teuthology.orchestra.run.smithi049.stdout:654 punch from 0x5d4ae6f to 0x5d54710, (0x98a1 bytes) 2024-04-24T18:07:46.169 INFO:teuthology.orchestra.run.smithi049.stdout:655 write 0x4f94460 thru 0x4f9c2e5 (0x7e86 bytes) 2024-04-24T18:07:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:656 writesame 0x807e781 thru 0x808ba80 data_size 0x20(0xd300 bytes) 2024-04-24T18:07:46.193 INFO:teuthology.orchestra.run.smithi049.stdout:657 writesame 0xa84c11e thru 0xa8566f5 data_size 0x57(0xa5d8 bytes) 2024-04-24T18:07:46.193 INFO:teuthology.orchestra.run.smithi049.stdout:660 read 0x4ff2842 thru 0x4ffe3c9 (0xbb88 bytes) 2024-04-24T18:07:46.201 INFO:teuthology.orchestra.run.smithi049.stdout:661 writesame 0xc16dfd9 thru 0xc17d99b data_size 0x1(0xf9c3 bytes) 2024-04-24T18:07:46.201 INFO:teuthology.orchestra.run.smithi049.stdout:662 read 0xb45e244 thru 0xb46659d (0x835a bytes) 2024-04-24T18:07:46.201 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.201 INFO:teuthology.orchestra.run.smithi049.stdout:664 punch from 0x97aa12b to 0x97aea56, (0x492b bytes) 2024-04-24T18:07:46.201 INFO:teuthology.orchestra.run.smithi049.stdout:665 write 0x6b5387 thru 0x6b5b0a (0x784 bytes) 2024-04-24T18:07:46.211 INFO:teuthology.orchestra.run.smithi049.stdout:668 trunc from 0xc17d99c to 0x3b48076 2024-04-24T18:07:46.220 INFO:teuthology.orchestra.run.smithi049.stdout:669 read 0x23bb07e thru 0x23c61a2 (0xb125 bytes) 2024-04-24T18:07:46.221 INFO:teuthology.orchestra.run.smithi049.stdout:670 punch from 0x1fc376c to 0x1fc922d, (0x5ac1 bytes) 2024-04-24T18:07:46.221 INFO:teuthology.orchestra.run.smithi049.stdout:673 write 0x11fe6b7 thru 0x12099d9 (0xb323 bytes) 2024-04-24T18:07:46.224 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:46.224 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:46.226 INFO:teuthology.orchestra.run.smithi049.stdout:674 trunc from 0x3b48076 to 0x1ce8111 2024-04-24T18:07:46.235 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:07:46.252 INFO:teuthology.orchestra.run.smithi049.stdout:675 writesame 0xd7d73a6 thru 0xd7e0dcc data_size 0x1f(0x9a27 bytes) 2024-04-24T18:07:46.266 INFO:teuthology.orchestra.run.smithi049.stdout:676 trunc from 0xd7e0dcd to 0x157d320 2024-04-24T18:07:46.270 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:46.278+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:46.279 INFO:teuthology.orchestra.run.smithi049.stdout:677 write 0x1a20be6 thru 0x1a2a59f (0x99ba bytes) 2024-04-24T18:07:46.296 INFO:teuthology.orchestra.run.smithi049.stdout:678 writesame 0x85fd341 thru 0x860a058 data_size 0x4(0xcd18 bytes) 2024-04-24T18:07:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:679 read 0x6e3000f thru 0x6e363b6 (0x63a8 bytes) 2024-04-24T18:07:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:680 read 0x59fc066 thru 0x5a00d3c (0x4cd7 bytes) 2024-04-24T18:07:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:681 read 0x2e893d thru 0x2f4058 (0xb71c bytes) 2024-04-24T18:07:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:682 read 0x716f3bf thru 0x717545d (0x609f bytes) 2024-04-24T18:07:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:684 writesame 0x47d980d thru 0x47e8e78 data_size 0x1c(0xf66c bytes) 2024-04-24T18:07:46.300 INFO:teuthology.orchestra.run.smithi049.stdout:685 read 0x4917d71 thru 0x4927616 (0xf8a6 bytes) 2024-04-24T18:07:46.300 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.300 INFO:teuthology.orchestra.run.smithi049.stdout:688 punch from 0x17b3ec5 to 0x17b7ff7, (0x4132 bytes) 2024-04-24T18:07:46.300 INFO:teuthology.orchestra.run.smithi049.stdout:690 read 0x217006f thru 0x217c612 (0xc5a4 bytes) 2024-04-24T18:07:46.300 INFO:teuthology.orchestra.run.smithi049.stdout:692 write 0x112a148 thru 0x1138e6d (0xed26 bytes) 2024-04-24T18:07:46.304 INFO:teuthology.orchestra.run.smithi049.stdout:695 trunc from 0x860a059 to 0xa4921c0 2024-04-24T18:07:46.309 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.313 INFO:teuthology.orchestra.run.smithi049.stdout:697 trunc from 0xa4921c0 to 0x7361c95 2024-04-24T18:07:46.326 INFO:teuthology.orchestra.run.smithi049.stdout:698 writesame 0x91a5d6e thru 0x91acfbf data_size 0x1(0x7252 bytes) 2024-04-24T18:07:46.334 INFO:teuthology.orchestra.run.smithi049.stdout:699 write 0xa9d4e29 thru 0xa9dd98a (0x8b62 bytes) 2024-04-24T18:07:46.337 INFO:teuthology.orchestra.run.smithi049.stdout:700 read 0x917944e thru 0x9183be8 (0xa79b bytes) 2024-04-24T18:07:46.337 INFO:teuthology.orchestra.run.smithi049.stdout:701 writesame 0x300353f thru 0x3004938 data_size 0x1(0x13fa bytes) 2024-04-24T18:07:46.346 INFO:teuthology.orchestra.run.smithi049.stdout:702 trunc from 0xa9dd98b to 0x65c72fa 2024-04-24T18:07:46.356 INFO:teuthology.orchestra.run.smithi049.stdout:703 read 0x41e7d29 thru 0x41f2816 (0xaaee bytes) 2024-04-24T18:07:46.356 INFO:teuthology.orchestra.run.smithi049.stdout:705 writesame 0x2a30ad0 thru 0x2a3cf5d data_size 0x2(0xc48e bytes) 2024-04-24T18:07:46.366 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:46.366 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:46.371 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:46.379+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:46.373 INFO:teuthology.orchestra.run.smithi049.stdout:706 trunc from 0x65c72fa to 0x9413c0d 2024-04-24T18:07:46.377 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:07:46.379 INFO:teuthology.orchestra.run.smithi049.stdout:708 punch from 0x3dfb931 to 0x3e012d0, (0x599f bytes) 2024-04-24T18:07:46.380 INFO:teuthology.orchestra.run.smithi049.stdout:709 read 0x3924e83 thru 0x3925243 (0x3c1 bytes) 2024-04-24T18:07:46.385 INFO:teuthology.orchestra.run.smithi049.stdout:710 write 0x95d0ef2 thru 0x95d6ba1 (0x5cb0 bytes) 2024-04-24T18:07:46.396 INFO:teuthology.orchestra.run.smithi049.stdout:711 write 0xb1699cc thru 0xb175752 (0xbd87 bytes) 2024-04-24T18:07:46.399 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.408 INFO:teuthology.orchestra.run.smithi049.stdout:715 trunc from 0xb175753 to 0x3895e99 2024-04-24T18:07:46.434 INFO:teuthology.orchestra.run.smithi049.stdout:716 writesame 0x97c5501 thru 0x97d2fe8 data_size 0x3c(0xdae8 bytes) 2024-04-24T18:07:46.434 INFO:teuthology.orchestra.run.smithi049.stdout:717 punch from 0x8b5d0c3 to 0x8b6304f, (0x5f8c bytes) 2024-04-24T18:07:46.434 INFO:teuthology.orchestra.run.smithi049.stdout:718 write 0x6b0ab1b thru 0x6b1a198 (0xf67e bytes) 2024-04-24T18:07:46.437 INFO:teuthology.orchestra.run.smithi049.stdout:719 read 0x95dcb5e thru 0x95ea8ed (0xdd90 bytes) 2024-04-24T18:07:46.445 INFO:teuthology.orchestra.run.smithi049.stdout:720 trunc from 0x97d2fe9 to 0x28ad2ec 2024-04-24T18:07:46.458 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.467 INFO:teuthology.orchestra.run.smithi049.stdout:722 write 0x5afc4f6 thru 0x5b05dd4 (0x98df bytes) 2024-04-24T18:07:46.470 INFO:teuthology.orchestra.run.smithi049.stdout:723 read 0x1457c6c thru 0x145ed11 (0x70a6 bytes) 2024-04-24T18:07:46.470 INFO:teuthology.orchestra.run.smithi049.stdout:724 write 0x45d1cf1 thru 0x45d4667 (0x2977 bytes) 2024-04-24T18:07:46.472 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:46.480+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:46.473 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.473 INFO:teuthology.orchestra.run.smithi049.stdout:729 punch from 0x4ffcae2 to 0x5007a15, (0xaf33 bytes) 2024-04-24T18:07:46.479 INFO:teuthology.orchestra.run.smithi049.stdout:730 writesame 0x5ee544a thru 0x5eeaeb8 data_size 0x1(0x5a6f bytes) 2024-04-24T18:07:46.479 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.479 INFO:teuthology.orchestra.run.smithi049.stdout:733 read 0xa4637e thru 0xa4b6d9 (0x535c bytes) 2024-04-24T18:07:46.480 INFO:teuthology.orchestra.run.smithi049.stdout:736 read 0x2037f14 thru 0x2044194 (0xc281 bytes) 2024-04-24T18:07:46.480 INFO:teuthology.orchestra.run.smithi049.stdout:737 trunc from 0x5eeaeb9 to 0x611f700 2024-04-24T18:07:46.487 INFO:teuthology.orchestra.run.smithi049.stdout:738 trunc from 0x611f700 to 0x3e1d7a4 2024-04-24T18:07:46.497 INFO:teuthology.orchestra.run.smithi049.stdout:739 punch from 0xaa99e2 to 0xab089e, (0x6ebc bytes) 2024-04-24T18:07:46.497 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.497 INFO:teuthology.orchestra.run.smithi049.stdout:742 read 0x1ab4d68 thru 0x1ac3ba0 (0xee39 bytes) 2024-04-24T18:07:46.498 INFO:teuthology.orchestra.run.smithi049.stdout:743 read 0xa94a51 thru 0xaa19b3 (0xcf63 bytes) 2024-04-24T18:07:46.498 INFO:teuthology.orchestra.run.smithi049.stdout:744 punch from 0x2c4b203 to 0x2c59027, (0xde24 bytes) 2024-04-24T18:07:46.498 INFO:teuthology.orchestra.run.smithi049.stdout:745 read 0x3363a85 thru 0x336d833 (0x9daf bytes) 2024-04-24T18:07:46.498 INFO:teuthology.orchestra.run.smithi049.stdout:746 read 0x1f5d183 thru 0x1f69928 (0xc7a6 bytes) 2024-04-24T18:07:46.498 INFO:teuthology.orchestra.run.smithi049.stdout:747 write 0x198fc59 thru 0x199e9bf (0xed67 bytes) 2024-04-24T18:07:46.507 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:46.507 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:46.509 INFO:teuthology.orchestra.run.smithi049.stdout:748 writesame 0x68c748a thru 0x68c9b23 data_size 0x3d(0x269a bytes) 2024-04-24T18:07:46.509 INFO:teuthology.orchestra.run.smithi049.stdout:752 write 0x1b8c37f thru 0x1b9c0e2 (0xfd64 bytes) 2024-04-24T18:07:46.518 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:07:46.525 INFO:teuthology.orchestra.run.smithi049.stdout:753 writesame 0xdda5461 thru 0xddb2af6 data_size 0x79(0xd696 bytes) 2024-04-24T18:07:46.525 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.525 INFO:teuthology.orchestra.run.smithi049.stdout:757 write 0xa959957 thru 0xa968960 (0xf00a bytes) 2024-04-24T18:07:46.528 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.529 INFO:teuthology.orchestra.run.smithi049.stdout:761 punch from 0x6fe235e to 0x6fe31c3, (0xe65 bytes) 2024-04-24T18:07:46.529 INFO:teuthology.orchestra.run.smithi049.stdout:762 read 0x4097cb5 thru 0x409b7b4 (0x3b00 bytes) 2024-04-24T18:07:46.529 INFO:teuthology.orchestra.run.smithi049.stdout:763 write 0xb196878 thru 0xb1a2aa0 (0xc229 bytes) 2024-04-24T18:07:46.532 INFO:teuthology.orchestra.run.smithi049.stdout:764 read 0x3d76003 thru 0x3d82686 (0xc684 bytes) 2024-04-24T18:07:46.532 INFO:teuthology.orchestra.run.smithi049.stdout:765 punch from 0xccd67a2 to 0xcce157e, (0xaddc bytes) 2024-04-24T18:07:46.532 INFO:teuthology.orchestra.run.smithi049.stdout:766 write 0x34f6db6 thru 0x34fee3b (0x8086 bytes) 2024-04-24T18:07:46.534 INFO:teuthology.orchestra.run.smithi049.stdout:767 read 0x854e0b1 thru 0x8559b74 (0xbac4 bytes) 2024-04-24T18:07:46.534 INFO:teuthology.orchestra.run.smithi049.stdout:768 write 0x89b5419 thru 0x89bf69d (0xa285 bytes) 2024-04-24T18:07:46.537 INFO:teuthology.orchestra.run.smithi049.stdout:769 write 0x1a538df thru 0x1a549c3 (0x10e5 bytes) 2024-04-24T18:07:46.550 INFO:teuthology.orchestra.run.smithi049.stdout:770 trunc from 0xddb2af7 to 0x3815c01 2024-04-24T18:07:46.564 INFO:teuthology.orchestra.run.smithi049.stdout:771 write 0x972a10 thru 0x97a23b (0x782c bytes) 2024-04-24T18:07:46.566 INFO:teuthology.orchestra.run.smithi049.stdout:773 read 0x190ee95 thru 0x1917e47 (0x8fb3 bytes) 2024-04-24T18:07:46.573 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:46.581+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:46.584 INFO:teuthology.orchestra.run.smithi049.stdout:774 write 0xe05b481 thru 0xe061ba5 (0x6725 bytes) 2024-04-24T18:07:46.587 INFO:teuthology.orchestra.run.smithi049.stdout:776 read 0x54db819 thru 0x54e22fa (0x6ae2 bytes) 2024-04-24T18:07:46.587 INFO:teuthology.orchestra.run.smithi049.stdout:777 read 0x31b2751 thru 0x31b40da (0x198a bytes) 2024-04-24T18:07:46.587 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.587 INFO:teuthology.orchestra.run.smithi049.stdout:779 read 0x7e8c633 thru 0x7e9a727 (0xe0f5 bytes) 2024-04-24T18:07:46.587 INFO:teuthology.orchestra.run.smithi049.stdout:780 writesame 0x840cd5b thru 0x84171ef data_size 0xd(0xa495 bytes) 2024-04-24T18:07:46.591 INFO:teuthology.orchestra.run.smithi049.stdout:782 read 0x3b26672 thru 0x3b29130 (0x2abf bytes) 2024-04-24T18:07:46.591 INFO:teuthology.orchestra.run.smithi049.stdout:783 read 0x15797e7 thru 0x1588244 (0xea5e bytes) 2024-04-24T18:07:46.591 INFO:teuthology.orchestra.run.smithi049.stdout:784 write 0x5b2b15e thru 0x5b376e1 (0xc584 bytes) 2024-04-24T18:07:46.597 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:787 write 0x96875bb thru 0x968ea3b (0x7481 bytes) 2024-04-24T18:07:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:788 read 0xd2a7c18 thru 0xd2b0f00 (0x92e9 bytes) 2024-04-24T18:07:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:789 read 0x7f99073 thru 0x7f99d49 (0xcd7 bytes) 2024-04-24T18:07:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:792 read 0x1144dd7 thru 0x114a8c5 (0x5aef bytes) 2024-04-24T18:07:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:793 read 0x59f2964 thru 0x5a01737 (0xedd4 bytes) 2024-04-24T18:07:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:794 write 0x15ecf7b thru 0x15f631d (0x93a3 bytes) 2024-04-24T18:07:46.600 INFO:teuthology.orchestra.run.smithi049.stdout:truncating to largest ever: 0xeacf705 2024-04-24T18:07:46.601 INFO:teuthology.orchestra.run.smithi049.stdout:795 trunc from 0xe061ba6 to 0xeacf705 2024-04-24T18:07:46.606 INFO:teuthology.orchestra.run.smithi049.stdout:796 read 0x72f7fc2 thru 0x72fa163 (0x21a2 bytes) 2024-04-24T18:07:46.606 INFO:teuthology.orchestra.run.smithi049.stdout:797 read 0x3808d90 thru 0x3808f9b (0x20c bytes) 2024-04-24T18:07:46.606 INFO:teuthology.orchestra.run.smithi049.stdout:798 write 0x3d8ed48 thru 0x3d91168 (0x2421 bytes) 2024-04-24T18:07:46.609 INFO:teuthology.orchestra.run.smithi049.stdout:799 writesame 0x6e059cd thru 0x6e05b07 data_size 0x9(0x13b bytes) 2024-04-24T18:07:46.611 INFO:teuthology.orchestra.run.smithi049.stdout:800 write 0x76af5f8 thru 0x76b1cf1 (0x26fa bytes) 2024-04-24T18:07:46.614 INFO:teuthology.orchestra.run.smithi049.stdout:802 punch from 0x4426bfc to 0x4433b2d, (0xcf31 bytes) 2024-04-24T18:07:46.614 INFO:teuthology.orchestra.run.smithi049.stdout:803 trunc from 0xeacf705 to 0xe24e2e3 2024-04-24T18:07:46.620 INFO:teuthology.orchestra.run.smithi049.stdout:804 writesame 0x9552084 thru 0x95542ab data_size 0x4(0x2228 bytes) 2024-04-24T18:07:46.624 INFO:teuthology.orchestra.run.smithi049.stdout:805 read 0x6a129c2 thru 0x6a16810 (0x3e4f bytes) 2024-04-24T18:07:46.624 INFO:teuthology.orchestra.run.smithi049.stdout:806 clone 5 order 22 su 16384 sc 3 2024-04-24T18:07:46.649 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:46.649 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:46.659 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:07:46.675 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:46.682+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:46.721 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7] out_osds: [1] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:07:46.721 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-24T18:07:46.776 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:46.784+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:46.783 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:46.783 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:46.793 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:07:46.878 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:46.886+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:46.958 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:46.958 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:46.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:07:46.977 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:46.985+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:47.067 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:07:47.078 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:47.087+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:47.131 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:47.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:07:47.183 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:47.191+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:47.210 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:47.228 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:47.247 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:47.265 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:47.282 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.0 to 1 (10000) 2024-04-24T18:07:47.282 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:47.291+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:47.287 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:47.292 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-24T18:07:47.309 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:47.312 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:47.312 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:47.323 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:07:47.332 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:47.352 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:47.354 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:47.409 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:47.417+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:47.486 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #3, image image_client.0-clone3 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent4 2024-04-24T18:07:47.486 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:47.494+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:47.492 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:47.492 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:47.492 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:47.492 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:47.503 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:07:47.587 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:07:47.588 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:47.596+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:47.667 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:47.667 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:47.668 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:47.677 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:07:47.718 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:47.726+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:47.724 INFO:teuthology.orchestra.run.smithi049.stdout:809 write 0x639410d thru 0x6399c63 (0x5b57 bytes) 2024-04-24T18:07:47.738 INFO:teuthology.orchestra.run.smithi049.stdout:812 trunc from 0xe24e2e3 to 0xa2f1504 2024-04-24T18:07:47.766 INFO:teuthology.orchestra.run.smithi049.stdout:813 punch from 0x7506411 to 0x750c771, (0x6360 bytes) 2024-04-24T18:07:47.766 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:47.766 INFO:teuthology.orchestra.run.smithi049.stdout:815 punch from 0x70f8ea9 to 0x7104caf, (0xbe06 bytes) 2024-04-24T18:07:47.766 INFO:teuthology.orchestra.run.smithi049.stdout:816 read 0x87d8e75 thru 0x87da369 (0x14f5 bytes) 2024-04-24T18:07:47.766 INFO:teuthology.orchestra.run.smithi049.stdout:817 writesame 0x57cf641 thru 0x57d0e9c data_size 0x2(0x185c bytes) 2024-04-24T18:07:47.786 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:47.786 INFO:teuthology.orchestra.run.smithi049.stdout:820 punch from 0x4be5259 to 0x4beb078, (0x5e1f bytes) 2024-04-24T18:07:47.786 INFO:teuthology.orchestra.run.smithi049.stdout:821 punch from 0x2013007 to 0x2016d0f, (0x3d08 bytes) 2024-04-24T18:07:47.786 INFO:teuthology.orchestra.run.smithi049.stdout:822 write 0x64fdc93 thru 0x650d39f (0xf70d bytes) 2024-04-24T18:07:47.789 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:47.797+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:47.793 INFO:teuthology.orchestra.run.smithi049.stdout:823 trunc from 0xa2f1504 to 0x6ce3f14 2024-04-24T18:07:47.827 INFO:teuthology.orchestra.run.smithi049.stdout:825 read 0x2b3b0f4 thru 0x2b4a208 (0xf115 bytes) 2024-04-24T18:07:47.828 INFO:teuthology.orchestra.run.smithi049.stdout:826 read 0x4410b82 thru 0x441bae1 (0xaf60 bytes) 2024-04-24T18:07:47.840 INFO:teuthology.orchestra.run.smithi049.stdout:827 write 0xd16fe8f thru 0xd17d2cf (0xd441 bytes) 2024-04-24T18:07:47.845 INFO:teuthology.orchestra.run.smithi049.stdout:829 punch from 0x4442d9d to 0x444b399, (0x85fc bytes) 2024-04-24T18:07:47.845 INFO:teuthology.orchestra.run.smithi049.stdout:831 write 0x92e3854 thru 0x92e4b7e (0x132b bytes) 2024-04-24T18:07:47.849 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:47.849 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:47.849 INFO:teuthology.orchestra.run.smithi049.stdout:834 write 0x61af125 thru 0x61b6dfd (0x7cd9 bytes) 2024-04-24T18:07:47.850 INFO:teuthology.orchestra.run.smithi049.stdout:836 write 0x8fa984a thru 0x8fb70d8 (0xd88f bytes) 2024-04-24T18:07:47.856 INFO:teuthology.orchestra.run.smithi049.stdout:838 punch from 0x2e5dc25 to 0x2e681d8, (0xa5b3 bytes) 2024-04-24T18:07:47.856 INFO:teuthology.orchestra.run.smithi049.stdout:840 read 0xcd1d585 thru 0xcd23df2 (0x686e bytes) 2024-04-24T18:07:47.856 INFO:teuthology.orchestra.run.smithi049.stdout:841 punch from 0xfa074 to 0xfdf14, (0x3ea0 bytes) 2024-04-24T18:07:47.858 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:47.858 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:47.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:47.858 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.247291128000001, 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.859 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:07:47.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:07:47.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:07:47.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:07:47.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:07:47.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:07:47.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:07:47.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:07:47.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.247222076, 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.247197661000001, 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.873 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:07:47.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:07:47.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:07:47.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:07:47.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.879 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.880 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.880 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.880 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:47.880 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.247174512000001, 2024-04-24T18:07:47.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:07:47.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:07:47.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.247093944, 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:07:47.891 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.247069134, 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.893 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:07:47.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:07:47.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:07:47.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:07:47.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.247049021999999, 2024-04-24T18:07:47.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.246978359, 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:07:47.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.246966585999999, 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.911 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:47.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:07:47.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:07:47.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.826983223999999, 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0070057069999999999, 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.917 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 132 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036533+0000", 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036535+0000", 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036539+0000", 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3459999999999998e-06 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036541+0000", 2024-04-24T18:07:47.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9520000000000001e-06 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037943+0000", 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001402401 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038021+0000", 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8071000000000005e-05 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038150+0000", 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012869199999999999 2024-04-24T18:07:47.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.039054+0000", 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00090387400000000004 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043447+0000", 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043929110000000002 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043497+0000", 2024-04-24T18:07:47.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0420999999999997e-05 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043539+0000", 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2255000000000001e-05 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.826680973999999, 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0068998769999999996, 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036839+0000", 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9739999999999998e-06 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036844+0000", 2024-04-24T18:07:47.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3079999999999998e-06 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036846+0000", 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4099999999999998e-06 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040072+0000", 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003225805 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040118+0000", 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6026999999999998e-05 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040191+0000", 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2983000000000002e-05 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042013+0000", 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001822118 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043702+0000", 2024-04-24T18:07:47.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0016890329999999999 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043721+0000", 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8754999999999999e-05 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043736+0000", 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4725e-05 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.928 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.826644757, 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.929 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:47.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:07:47.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.934 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.826597575000001, 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:07:47.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:07:47.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:07:47.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:07:47.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.826572454000001, 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:47.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:07:47.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:07:47.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.946 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.82652032, 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:07:47.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:07:47.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:07:47.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:07:47.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.826456856, 2024-04-24T18:07:47.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.315286557, 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073643390000000001, 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 156 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548239+0000", 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8899999999999996e-06 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548246+0000", 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9370000000000004e-06 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548259+0000", 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:07:47.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549586+0000", 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001326743 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549641+0000", 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5856999999999997e-05 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549713+0000", 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1970000000000004e-05 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550899+0000", 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001185567 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555532+0000", 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0046325760000000002 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555573+0000", 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1730000000000002e-05 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555595+0000", 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1324000000000001e-05 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.963 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.315251184000001, 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.315226157000001, 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:07:47.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:07:47.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:07:47.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.975 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.31514692, 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.976 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:07:47.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.982 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:47.982 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:47.982 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:47.984 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:07:47.985 INFO:teuthology.orchestra.run.smithi049.stdout:843 trunc from 0xd17d2d0 to 0x30f3aa7 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:846 write 0x468975d thru 0x4698b45 (0xf3e9 bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:847 writesame 0x560334f thru 0x5603d7e data_size 0x8(0xa30 bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:848 write 0xbe91c58 thru 0xbe960b4 (0x445d bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:850 write 0x4fe5944 thru 0x4febe6d (0x652a bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:851 read 0x3149959 thru 0x3150594 (0x6c3c bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:855 punch from 0xa33996b to 0xa33ef0d, (0x55a2 bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:856 punch from 0x8ce557 to 0x8d4264, (0x5d0d bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:858 punch from 0x94db8aa to 0x94e61c9, (0xa91f bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:861 trunc from 0xbe960b5 to 0xe97d9a0 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:862 read 0x20d7c7d thru 0x20e1dca (0xa14e bytes) 2024-04-24T18:07:47.986 INFO:teuthology.orchestra.run.smithi049.stdout:863 punch from 0xa9a644e to 0xa9b264a, (0xc1fc bytes) 2024-04-24T18:07:47.987 INFO:teuthology.orchestra.run.smithi049.stdout:864 trunc from 0xe97d9a0 to 0xcce6d54 2024-04-24T18:07:47.987 INFO:teuthology.orchestra.run.smithi049.stdout:865 trunc from 0xcce6d54 to 0xd103063 2024-04-24T18:07:47.987 INFO:teuthology.orchestra.run.smithi049.stdout:866 read 0xc93035 thru 0xc9d2f3 (0xa2bf bytes) 2024-04-24T18:07:47.987 INFO:teuthology.orchestra.run.smithi049.stdout:867 writesame 0x1b5b9c thru 0x1c1e03 data_size 0x4(0xc268 bytes) 2024-04-24T18:07:47.987 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:47.991+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:48.018 INFO:teuthology.orchestra.run.smithi049.stdout:868 trunc from 0xd103063 to 0x2b0586a 2024-04-24T18:07:48.057 INFO:teuthology.orchestra.run.smithi049.stdout:870 punch from 0x1a09388 to 0x1a107f6, (0x746e bytes) 2024-04-24T18:07:48.057 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:48.069 INFO:teuthology.orchestra.run.smithi049.stdout:872 writesame 0x8c43dff thru 0x8c4ef21 data_size 0x1(0xb123 bytes) 2024-04-24T18:07:48.069 INFO:teuthology.orchestra.run.smithi049.stdout:873 read 0xcf0f5f thru 0xcf13ae (0x450 bytes) 2024-04-24T18:07:48.069 INFO:teuthology.orchestra.run.smithi049.stdout:874 read 0x4ee48b7 thru 0x4ee680a (0x1f54 bytes) 2024-04-24T18:07:48.069 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:48.081 INFO:teuthology.orchestra.run.smithi049.stdout:877 write 0xe28a749 thru 0xe28bfd0 (0x1888 bytes) 2024-04-24T18:07:48.083 INFO:teuthology.orchestra.run.smithi049.stdout:878 write 0x18d8a91 thru 0x18dcd58 (0x42c8 bytes) 2024-04-24T18:07:48.084 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:48.092+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:48.110 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:48.110 INFO:teuthology.orchestra.run.smithi049.stdout:882 read 0xa190181 thru 0xa19182b (0x16ab bytes) 2024-04-24T18:07:48.110 INFO:teuthology.orchestra.run.smithi049.stdout:883 writesame 0x2d7bebb thru 0x2d849d3 data_size 0x1(0x8b19 bytes) 2024-04-24T18:07:48.118 INFO:teuthology.orchestra.run.smithi049.stdout:884 read 0x80955ba thru 0x809e1a0 (0x8be7 bytes) 2024-04-24T18:07:48.119 INFO:teuthology.orchestra.run.smithi049.stdout:885 punch from 0x955d077 to 0x955f1e4, (0x216d bytes) 2024-04-24T18:07:48.119 INFO:teuthology.orchestra.run.smithi049.stdout:886 write 0xdccb944 thru 0xdccc7c5 (0xe82 bytes) 2024-04-24T18:07:48.122 INFO:teuthology.orchestra.run.smithi049.stdout:887 read 0x6dec047 thru 0x6dee62b (0x25e5 bytes) 2024-04-24T18:07:48.122 INFO:teuthology.orchestra.run.smithi049.stdout:888 read 0x607edc8 thru 0x6081615 (0x284e bytes) 2024-04-24T18:07:48.127 INFO:teuthology.orchestra.run.smithi049.stdout:889 write 0xe8e3266 thru 0xe8eeb06 (0xb8a1 bytes) 2024-04-24T18:07:48.132 INFO:teuthology.orchestra.run.smithi049.stdout:892 clone 6 order 19 su 2048 sc 5 2024-04-24T18:07:48.149 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:48.149 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:48.149 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:48.149 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:48.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:07:48.202 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:48.209+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:48.285 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:48.294+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:48.329 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:48.329 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:48.330 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:48.330 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:48.339 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:07:48.386 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:48.394+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:48.469 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:1821 4.1e 4:7e3169c0:::rbd_data.1174cab6075.00000000000000a7:head [set-alloc-hint object_size 1048576 write_size 1048576,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e33)", 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:42.648686+0000", 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.8279957500000004, 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.029350831000000001, 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.470 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 1821 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648686+0000", 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648686+0000", 2024-04-24T18:07:48.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648685+0000", 2024-04-24T18:07:48.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:07:48.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648692+0000", 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4340000000000003e-06 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648707+0000", 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4857e-05 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648714+0000", 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1119999999999998e-06 2024-04-24T18:07:48.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648761+0000", 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6959999999999998e-05 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.648847+0000", 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5464999999999996e-05 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.653108+0000", 2024-04-24T18:07:48.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0042610640000000002 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.677921+0000", 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.024812873999999999 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.677964+0000", 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3367e-05 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.678007+0000", 2024-04-24T18:07:48.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2809000000000003e-05 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:42.678037+0000", 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0170000000000001e-05 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5669 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1507328~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984407+0000", 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49227528100000001, 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013207637, 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5669 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.477 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984407+0000", 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984407+0000", 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984406+0000", 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984408+0000", 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2520000000000002e-06 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984415+0000", 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.207e-06 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984418+0000", 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6389999999999998e-06 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997444+0000", 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013026078 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997521+0000", 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6285999999999994e-05 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997614+0000", 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3535000000000005e-05 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5670 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1556480~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984438+0000", 2024-04-24T18:07:48.481 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49224344599999997, 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.01333077, 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5670 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984438+0000", 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984438+0000", 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984438+0000", 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984441+0000", 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.892e-06 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984449+0000", 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0339999999999997e-06 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984453+0000", 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6789999999999996e-06 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997653+0000", 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013199486999999999 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997702+0000", 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9191999999999999e-05 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997769+0000", 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6946000000000006e-05 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5671 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1605632~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984459+0000", 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49222299600000002, 2024-04-24T18:07:48.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013528396999999999, 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5671 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.488 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984459+0000", 2024-04-24T18:07:48.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984459+0000", 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984458+0000", 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984461+0000", 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984467+0000", 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8279999999999999e-06 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984473+0000", 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:07:48.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997808+0000", 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.01333463 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997883+0000", 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4738000000000002e-05 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.997987+0000", 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104357 2024-04-24T18:07:48.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5672 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1654784~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984481+0000", 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49220122399999999, 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013682891000000001, 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5672 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984481+0000", 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984481+0000", 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984480+0000", 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984483+0000", 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8009999999999999e-06 2024-04-24T18:07:48.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984493+0000", 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0764999999999999e-05 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984498+0000", 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3429999999999998e-06 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998023+0000", 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013525224000000001 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998071+0000", 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7979000000000003e-05 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998163+0000", 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2417000000000006e-05 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5673 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1703936~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984503+0000", 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.492178691, 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013805817999999999, 2024-04-24T18:07:48.498 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5673 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984503+0000", 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984503+0000", 2024-04-24T18:07:48.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984502+0000", 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984505+0000", 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7389999999999999e-06 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984513+0000", 2024-04-24T18:07:48.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9279999999999993e-06 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984517+0000", 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9829999999999998e-06 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998193+0000", 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013675857 2024-04-24T18:07:48.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998245+0000", 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2018999999999998e-05 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998309+0000", 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3912999999999996e-05 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.502 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5674 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1753088~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984522+0000", 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49216019100000002, 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013918082, 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.503 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5674 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984522+0000", 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984522+0000", 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984521+0000", 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984524+0000", 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.306e-06 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984530+0000", 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2380000000000002e-06 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984535+0000", 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8620000000000002e-06 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998336+0000", 2024-04-24T18:07:48.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013801305 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998376+0000", 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0216000000000003e-05 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998440+0000", 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3504999999999996e-05 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.507 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.509 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5675 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1802240~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984539+0000", 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.492142358, 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014444494, 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.510 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5675 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984539+0000", 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984539+0000", 2024-04-24T18:07:48.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984539+0000", 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984541+0000", 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3300000000000001e-06 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984548+0000", 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3690000000000003e-06 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984552+0000", 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9620000000000004e-06 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998466+0000", 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.01391425 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998539+0000", 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2787999999999995e-05 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.998984+0000", 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00044518200000000002 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5676 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1851392~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984556+0000", 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49212559299999997, 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014609291999999999, 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.515 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5676 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984556+0000", 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984556+0000", 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984556+0000", 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984558+0000", 2024-04-24T18:07:48.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4049999999999998e-06 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984572+0000", 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3434e-05 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984576+0000", 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4259999999999996e-06 2024-04-24T18:07:48.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999015+0000", 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0144385 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999085+0000", 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0908000000000006e-05 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999165+0000", 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0030000000000005e-05 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5677 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1900544~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984581+0000", 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49210062500000001, 2024-04-24T18:07:48.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014736775000000001, 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5677 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984581+0000", 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984581+0000", 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984581+0000", 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984584+0000", 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.506e-06 2024-04-24T18:07:48.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984592+0000", 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7849999999999992e-06 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984596+0000", 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3100000000000002e-06 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999183+0000", 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014586211 2024-04-24T18:07:48.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999254+0000", 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1318999999999999e-05 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999318+0000", 2024-04-24T18:07:48.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4053999999999994e-05 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5678 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1949696~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984601+0000", 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49208047900000002, 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014869195, 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5678 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984601+0000", 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984601+0000", 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984601+0000", 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984603+0000", 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984612+0000", 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3019999999999995e-06 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984616+0000", 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0820000000000001e-06 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999336+0000", 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014720165 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999404+0000", 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7667000000000001e-05 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999470+0000", 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6996999999999996e-05 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.529 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5679 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 1998848~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984621+0000", 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49206118999999998, 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014998466, 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5679 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984621+0000", 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984621+0000", 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984620+0000", 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984623+0000", 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4329999999999998e-06 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984631+0000", 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4849999999999992e-06 2024-04-24T18:07:48.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984635+0000", 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2660000000000003e-06 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999487+0000", 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014851826 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999556+0000", 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9352999999999998e-05 2024-04-24T18:07:48.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999619+0000", 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2571999999999997e-05 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5680 4.6 4:66dd96ed:::rbd_data.1174bc8f97cd.0000000000000003:8 [read 2048000~16384] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.984662+0000", 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49202025199999999, 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.015641097999999999, 2024-04-24T18:07:48.534 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5680 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984662+0000", 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984662+0000", 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984661+0000", 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984665+0000", 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6449999999999998e-06 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984672+0000", 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5109999999999997e-06 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.984677+0000", 2024-04-24T18:07:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2089999999999996e-06 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.000124+0000", 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.015446435 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.000210+0000", 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5908999999999994e-05 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.000303+0000", 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2898000000000004e-05 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:5767 4.16 4:688acc01:::rbd_object_map.11749ffb1524:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:47.986162+0000", 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.49052004599999999, 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.016530998000000002, 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 5767 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.986162+0000", 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.986162+0000", 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.986161+0000", 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.986167+0000", 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0639999999999999e-06 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.986182+0000", 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5141e-05 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.986189+0000", 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0040000000000002e-06 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999638+0000", 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013449001 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:47.999724+0000", 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6429999999999997e-05 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.000010+0000", 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028530800000000001 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.001835+0000", 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018255260000000001 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.002626+0000", 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00079133600000000001 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.002655+0000", 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8430999999999999e-05 2024-04-24T18:07:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.002693+0000", 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7805000000000001e-05 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6547 4.6 4:666f8fe7:::rbd_data.1174bc8f97cd.0000000000000009:8 [read 3932160~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.040905+0000", 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.435777254, 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013256294, 2024-04-24T18:07:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6547 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040905+0000", 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040905+0000", 2024-04-24T18:07:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040904+0000", 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040906+0000", 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4090000000000001e-06 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040912+0000", 2024-04-24T18:07:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0100000000000001e-06 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040916+0000", 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.912e-06 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054085+0000", 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013168361999999999 2024-04-24T18:07:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054120+0000", 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5329e-05 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054161+0000", 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0843000000000002e-05 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6548 4.6 4:666f8fe7:::rbd_data.1174bc8f97cd.0000000000000009:8 [read 3981312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.040945+0000", 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.43573645100000002, 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013337801999999999, 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6548 2024-04-24T18:07:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040945+0000", 2024-04-24T18:07:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040945+0000", 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040944+0000", 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040950+0000", 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.062e-06 2024-04-24T18:07:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040962+0000", 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2447e-05 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040968+0000", 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.443e-06 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054187+0000", 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013218334999999999 2024-04-24T18:07:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054237+0000", 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9976999999999999e-05 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054283+0000", 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6424000000000001e-05 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6549 4.6 4:666f8fe7:::rbd_data.1174bc8f97cd.0000000000000009:8 [read 4030464~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.040978+0000", 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.435703322, 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.01341607, 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6549 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040978+0000", 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040978+0000", 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040978+0000", 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040981+0000", 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9129999999999999e-06 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040988+0000", 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3980000000000002e-06 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040993+0000", 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6759999999999998e-06 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054302+0000", 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013309054000000001 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054347+0000", 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5108000000000002e-05 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054395+0000", 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7377999999999999e-05 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6550 4.6 4:666f8fe7:::rbd_data.1174bc8f97cd.0000000000000009:8 [read 4079616~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.040998+0000", 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.43568380099999998, 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013505624000000001, 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6550 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040998+0000", 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040998+0000", 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.040998+0000", 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041005+0000", 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2699999999999999e-06 2024-04-24T18:07:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041012+0000", 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1359999999999999e-06 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041016+0000", 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4009999999999999e-06 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054413+0000", 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013396867 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054459+0000", 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5401999999999997e-05 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054504+0000", 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4916000000000002e-05 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6551 4.6 4:666f8fe7:::rbd_data.1174bc8f97cd.0000000000000009:8 [read 4128768~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.041021+0000", 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.43566043300000001, 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013604814999999999, 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6551 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041021+0000", 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041021+0000", 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041021+0000", 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041025+0000", 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9570000000000004e-06 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041032+0000", 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7889999999999997e-06 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041036+0000", 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2849999999999996e-06 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054524+0000", 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013488484 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054574+0000", 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9298999999999997e-05 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.054626+0000", 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2447000000000001e-05 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6552 4.6 4:666f8fe7:::rbd_data.1174bc8f97cd.0000000000000009:8 [read 4177920~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:07:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.041041+0000", 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.43564112300000002, 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014283966, 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6552 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041041+0000", 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041041+0000", 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041040+0000", 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041043+0000", 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5519999999999999e-06 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041049+0000", 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.229e-06 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.041053+0000", 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8190000000000002e-06 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.055188+0000", 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014135440000000001 2024-04-24T18:07:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.055256+0000", 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7409000000000005e-05 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.055325+0000", 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8897000000000001e-05 2024-04-24T18:07:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:48.574 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:48.575 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:07:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:07:48.576 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.2 to 1 (10000) 2024-04-24T18:07:48.576 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:48.583+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:48.589 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-24T18:07:48.631 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone5 from 0xe8eeb07 (overlap 0x2b0586a) to 0x1e1d77d 2024-04-24T18:07:48.676 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:48.684+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:48.727 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #4, image image_client.0-clone4 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent5 2024-04-24T18:07:48.775 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:48.775 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:48.775 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:48.775 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:48.776 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:48.785+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:48.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:07:48.877 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:48.885+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:48.930 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:48.931 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:48.931 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:48.931 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:48.931 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:48.942 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:07:48.978 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:48.986+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:48.995 INFO:teuthology.orchestra.run.smithi049.stdout:893 read 0x4a60169 thru 0x4a64118 (0x3fb0 bytes) 2024-04-24T18:07:48.996 INFO:teuthology.orchestra.run.smithi049.stdout:894 writesame 0xb97a1dd thru 0xb981702 data_size 0x5(0x7526 bytes) 2024-04-24T18:07:49.007 INFO:teuthology.orchestra.run.smithi049.stdout:896 read 0x93b7177 thru 0x93c6d8d (0xfc17 bytes) 2024-04-24T18:07:49.007 INFO:teuthology.orchestra.run.smithi049.stdout:897 read 0x9aa363e thru 0x9aaf301 (0xbcc4 bytes) 2024-04-24T18:07:49.008 INFO:teuthology.orchestra.run.smithi049.stdout:898 write 0xcc9046c thru 0xcc9bd95 (0xb92a bytes) 2024-04-24T18:07:49.017 INFO:teuthology.orchestra.run.smithi049.stdout:899 write 0x3bea8c0 thru 0x3bea96f (0xb0 bytes) 2024-04-24T18:07:49.021 INFO:teuthology.orchestra.run.smithi049.stdout:900 read 0x77af0d7 thru 0x77bcd9f (0xdcc9 bytes) 2024-04-24T18:07:49.022 INFO:teuthology.orchestra.run.smithi049.stdout:901 read 0xa7edc73 thru 0xa7f22ba (0x4648 bytes) 2024-04-24T18:07:49.022 INFO:teuthology.orchestra.run.smithi049.stdout:902 punch from 0xe51051 to 0xe5ab3b, (0x9aea bytes) 2024-04-24T18:07:49.022 INFO:teuthology.orchestra.run.smithi049.stdout:903 write 0x3c0882 thru 0x3ccc08 (0xc387 bytes) 2024-04-24T18:07:49.078 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:49.086+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:49.090 INFO:teuthology.orchestra.run.smithi049.stdout:904 writesame 0x1b7eb24 thru 0x1b83c1b data_size 0x4(0x50f8 bytes) 2024-04-24T18:07:49.179 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:49.187+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:49.182 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:20 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.767490908999999, 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001123017, 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422414+0000", 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000112488 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422424+0000", 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0118e-05 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422431+0000", 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9319999999999999e-06 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422477+0000", 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5738999999999997e-05 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422501+0000", 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4032e-05 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.423397+0000", 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089551500000000005 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.423425+0000", 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8775000000000001e-05 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.757171433, 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001094629, 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.187 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432621+0000", 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432627+0000", 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5160000000000002e-06 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432635+0000", 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3669999999999997e-06 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432639+0000", 2024-04-24T18:07:49.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9179999999999996e-06 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432678+0000", 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8627999999999997e-05 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432701+0000", 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3246000000000001e-05 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433677+0000", 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097568000000000004 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433716+0000", 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9765999999999998e-05 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.756620499, 2024-04-24T18:07:49.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00093691599999999999, 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433172+0000", 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433240+0000", 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7769999999999997e-05 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433253+0000", 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3118999999999999e-05 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433259+0000", 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3090000000000001e-06 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433287+0000", 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7813e-05 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433300+0000", 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3004e-05 2024-04-24T18:07:49.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.434087+0000", 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00078750999999999995 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.434110+0000", 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2198999999999999e-05 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:54 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-04-24T18:07:49.195 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.741061340000002, 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014283620000000001, 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448731+0000", 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448738+0000", 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4590000000000003e-06 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448751+0000", 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2833000000000001e-05 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448757+0000", 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448800+0000", 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3282999999999999e-05 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448823+0000", 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2731999999999999e-05 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.450119+0000", 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001295942 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.450160+0000", 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1709999999999999e-05 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.199 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:57 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.737792575, 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00095936700000000003, 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452000+0000", 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452057+0000", 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7161999999999999e-05 2024-04-24T18:07:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452067+0000", 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0278999999999999e-05 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452072+0000", 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9429999999999999e-06 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452106+0000", 2024-04-24T18:07:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4023000000000003e-05 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452128+0000", 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1145000000000002e-05 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452929+0000", 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00080152800000000003 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452960+0000", 2024-04-24T18:07:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0939000000000003e-05 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.730535047, 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0009994750000000001, 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459262+0000", 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1609999999999996e-06 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459270+0000", 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5989999999999996e-06 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459274+0000", 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8539999999999996e-06 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459306+0000", 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1129e-05 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459317+0000", 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1785e-05 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.460227+0000", 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090945999999999996 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.460258+0000", 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0899999999999999e-05 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:71 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'45, mlcod=16'43)", 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.728301372000001, 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00093247299999999996, 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461491+0000", 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461555+0000", 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3937999999999997e-05 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461565+0000", 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4800000000000007e-06 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461569+0000", 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5340000000000001e-06 2024-04-24T18:07:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461616+0000", 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6587e-05 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461640+0000", 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4130000000000001e-05 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462380+0000", 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074047800000000003 2024-04-24T18:07:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462424+0000", 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3980000000000002e-05 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:72 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'46, mlcod=16'43)", 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.727843310000001, 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010010850000000001, 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461949+0000", 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462032+0000", 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2152999999999998e-05 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462044+0000", 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2170999999999999e-05 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462051+0000", 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9600000000000003e-06 2024-04-24T18:07:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462081+0000", 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9788e-05 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462094+0000", 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3910999999999999e-05 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462904+0000", 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081001000000000001 2024-04-24T18:07:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462951+0000", 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6597000000000001e-05 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:83 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.720412386, 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001099135, 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469380+0000", 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469387+0000", 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5220000000000004e-06 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469398+0000", 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1076e-05 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469405+0000", 2024-04-24T18:07:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4699999999999996e-06 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469441+0000", 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5429999999999998e-05 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469461+0000", 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9828999999999999e-05 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470443+0000", 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098290399999999994 2024-04-24T18:07:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470480+0000", 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6544000000000001e-05 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:84 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.719876946999999, 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00094213999999999995, 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469980+0000", 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4336000000000002e-05 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469991+0000", 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1191e-05 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470001+0000", 2024-04-24T18:07:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7680000000000003e-06 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470036+0000", 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5265000000000002e-05 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470056+0000", 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9646000000000001e-05 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470828+0000", 2024-04-24T18:07:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077209399999999997 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470858+0000", 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0639000000000003e-05 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:86 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.718709953000001, 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010291390000000001, 2024-04-24T18:07:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471089+0000", 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3430000000000001e-06 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471104+0000", 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4617000000000001e-05 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471108+0000", 2024-04-24T18:07:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6430000000000003e-06 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471168+0000", 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9917000000000002e-05 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471187+0000", 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9147000000000002e-05 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.472073+0000", 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00088590000000000001 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.472112+0000", 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999999e-05 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:89 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'59, mlcod=16'57)", 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.715918142, 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001052597, 2024-04-24T18:07:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473874+0000", 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473941+0000", 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6638999999999997e-05 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473950+0000", 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.234e-06 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473955+0000", 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5179999999999998e-06 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473991+0000", 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5689000000000003e-05 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474013+0000", 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.209e-05 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474889+0000", 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00087645799999999999 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474928+0000", 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8591999999999999e-05 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:104 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.573571951000002, 2024-04-24T18:07:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097261199999999998, 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616220+0000", 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616228+0000", 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1489999999999997e-06 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616239+0000", 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1029000000000001e-05 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616244+0000", 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0340000000000004e-06 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616274+0000", 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9957000000000001e-05 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616290+0000", 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6035000000000001e-05 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617130+0000", 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084017 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617194+0000", 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4188000000000003e-05 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:105 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.573032666, 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001106934, 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616760+0000", 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616765+0000", 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2839999999999997e-06 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616775+0000", 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7229999999999999e-06 2024-04-24T18:07:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616780+0000", 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7620000000000003e-06 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616818+0000", 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8637999999999998e-05 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616839+0000", 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0279e-05 2024-04-24T18:07:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617845+0000", 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010063089999999999 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617868+0000", 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.249e-05 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.572449583000001, 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010242809999999999, 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617343+0000", 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617408+0000", 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4926999999999997e-05 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617420+0000", 2024-04-24T18:07:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1977999999999999e-05 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617423+0000", 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.507e-06 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617458+0000", 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4999999999999997e-05 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617477+0000", 2024-04-24T18:07:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8902999999999998e-05 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.618336+0000", 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000859044 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.618368+0000", 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1656999999999999e-05 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:109 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.570507392, 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099814200000000004, 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619285+0000", 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619338+0000", 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2568999999999998e-05 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619347+0000", 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2350000000000005e-06 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619355+0000", 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8199999999999997e-06 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619377+0000", 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2169999999999999e-05 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619388+0000", 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0569e-05 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620251+0000", 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000863162 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620284+0000", 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3145999999999997e-05 2024-04-24T18:07:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'68)", 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.568909679000001, 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097267399999999998, 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620883+0000", 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620950+0000", 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7145999999999997e-05 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620958+0000", 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6240000000000002e-06 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620961+0000", 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3790000000000002e-06 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620992+0000", 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0784999999999997e-05 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621011+0000", 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8811999999999999e-05 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621819+0000", 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00080833000000000001 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621856+0000", 2024-04-24T18:07:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7004e-05 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 28.561570839000002, 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098543200000000006, 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628279+0000", 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6913000000000002e-05 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628289+0000", 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8539999999999992e-06 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628293+0000", 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4800000000000003e-06 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628329+0000", 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5945000000000002e-05 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628349+0000", 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0367999999999999e-05 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.629181+0000", 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00083184899999999996 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.629208+0000", 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6676e-05 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:154 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 27.640998777, 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001000143, 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548799+0000", 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9579999999999998e-06 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548808+0000", 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7220000000000005e-06 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548813+0000", 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0690000000000001e-06 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548846+0000", 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3419e-05 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548857+0000", 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.094e-05 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549758+0000", 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090098800000000003 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549795+0000", 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6531e-05 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:155 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 27.640601792999998, 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096950400000000005, 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549275+0000", 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4623999999999995e-05 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549283+0000", 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590000000000002e-06 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549292+0000", 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3340000000000008e-06 2024-04-24T18:07:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549331+0000", 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8844999999999998e-05 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549351+0000", 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9791000000000001e-05 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.550128+0000", 2024-04-24T18:07:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077682700000000001 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.550161+0000", 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3318999999999997e-05 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:49.265 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:49.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:07:49.297 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:49.297 INFO:teuthology.orchestra.run.smithi049.stdout:908 write 0x984e222 thru 0x98520e5 (0x3ec4 bytes) 2024-04-24T18:07:49.297 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:49.305+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:49.307 INFO:teuthology.orchestra.run.smithi049.stdout:910 trunc from 0xe8eeb07 to 0xdefe85e 2024-04-24T18:07:49.368 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:07:49.390 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:49.391 INFO:teuthology.orchestra.run.smithi049.stdout:912 writesame 0x184c544 thru 0x1857f32 data_size 0x1(0xb9ef bytes) 2024-04-24T18:07:49.391 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:49.398+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:49.455 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:49.455 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:49.455 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:49.456 INFO:teuthology.orchestra.run.smithi049.stdout:913 write 0x3a214c7 thru 0x3a2a802 (0x933c bytes) 2024-04-24T18:07:49.467 INFO:teuthology.orchestra.run.smithi049.stdout:914 write 0xa21514 thru 0xa29a31 (0x851e bytes) 2024-04-24T18:07:49.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:07:49.510 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:49.518+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:49.539 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:49.539 INFO:teuthology.orchestra.run.smithi049.stdout:917 read 0x729e42b thru 0x729f782 (0x1358 bytes) 2024-04-24T18:07:49.539 INFO:teuthology.orchestra.run.smithi049.stdout:918 read 0x32a59ee thru 0x32a67fd (0xe10 bytes) 2024-04-24T18:07:49.539 INFO:teuthology.orchestra.run.smithi049.stdout:921 writesame 0x4476f thru 0x4b339 data_size 0x1(0x6bcb bytes) 2024-04-24T18:07:49.570 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:49.582 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:49.590+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:49.594 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.3 to 1 (10000) 2024-04-24T18:07:49.610 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:49.611 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-24T18:07:49.627 INFO:teuthology.orchestra.run.smithi049.stdout:922 write 0xd746e4d thru 0xd74b8bc (0x4a70 bytes) 2024-04-24T18:07:49.638 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:49.679 INFO:teuthology.orchestra.run.smithi049.stdout:923 punch from 0x4ad133f to 0x4ad3a38, (0x26f9 bytes) 2024-04-24T18:07:49.679 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:49.683 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:49.691+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:49.685 INFO:teuthology.orchestra.run.smithi049.stdout:925 write 0xe44c0da thru 0xe455dd7 (0x9cfe bytes) 2024-04-24T18:07:49.692 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:49.692 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:49.694 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:49.697 INFO:teuthology.orchestra.run.smithi049.stdout:927 writesame 0xe65f8b8 thru 0xe663c3c data_size 0x1(0x4385 bytes) 2024-04-24T18:07:49.698 INFO:teuthology.orchestra.run.smithi049.stdout:928 writesame 0x8e1efbd thru 0x8e2ab38 data_size 0x1(0xbb7c bytes) 2024-04-24T18:07:49.706 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:07:49.711 INFO:teuthology.orchestra.run.smithi049.stdout:929 write 0x257727c thru 0x2581546 (0xa2cb bytes) 2024-04-24T18:07:49.731 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:49.767 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:49.785 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:49.793+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:49.797 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:49.807 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:49.808 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:49.808 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:49.840 INFO:teuthology.orchestra.run.smithi049.stdout:930 write 0xa138f6c thru 0xa13e5de (0x5673 bytes) 2024-04-24T18:07:49.852 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:49.852 INFO:teuthology.orchestra.run.smithi049.stdout:932 punch from 0x131e0dd to 0x131e336, (0x259 bytes) 2024-04-24T18:07:49.852 INFO:teuthology.orchestra.run.smithi049.stdout:933 punch from 0x8499eb2 to 0x84a4f10, (0xb05e bytes) 2024-04-24T18:07:49.856 INFO:teuthology.orchestra.run.smithi049.stdout:934 trunc from 0xe663c3d to 0xa8f51fe 2024-04-24T18:07:49.885 INFO:teuthology.orchestra.run.smithi049.stdout:935 write 0x35ac09e thru 0x35b7b8b (0xbaee bytes) 2024-04-24T18:07:49.886 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:49.894+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:49.893 INFO:teuthology.orchestra.run.smithi049.stdout:938 write 0xa1dcf45 thru 0xa1e397f (0x6a3b bytes) 2024-04-24T18:07:49.895 INFO:teuthology.orchestra.run.smithi049.stdout:939 writesame 0x2b98729 thru 0x2b9afdf data_size 0x1(0x28b7 bytes) 2024-04-24T18:07:49.915 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:49.915 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:49.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:07:49.988 INFO:teuthology.orchestra.run.smithi049.stdout:941 write 0x9f753ee thru 0x9f7f075 (0x9c88 bytes) 2024-04-24T18:07:49.989 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:49.996+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:50.000 INFO:teuthology.orchestra.run.smithi049.stdout:942 write 0x64a8b86 thru 0x64b6604 (0xda7f bytes) 2024-04-24T18:07:50.013 INFO:teuthology.orchestra.run.smithi049.stdout:943 read 0x7793832 thru 0x779499b (0x116a bytes) 2024-04-24T18:07:50.013 INFO:teuthology.orchestra.run.smithi049.stdout:944 write 0x2a2b558 thru 0x2a31bcb (0x6674 bytes) 2024-04-24T18:07:50.088 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:50.096+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:50.106 INFO:teuthology.orchestra.run.smithi049.stdout:945 trunc from 0xa8f51fe to 0xb5ff455 2024-04-24T18:07:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:947 punch from 0x4e82c45 to 0x4e8319d, (0x558 bytes) 2024-04-24T18:07:50.112 INFO:teuthology.orchestra.run.smithi049.stdout:948 write 0xabdb8cd thru 0xabe0c6d (0x53a1 bytes) 2024-04-24T18:07:50.119 INFO:teuthology.orchestra.run.smithi049.stdout:949 read 0x19f8364 thru 0x1a07269 (0xef06 bytes) 2024-04-24T18:07:50.121 INFO:teuthology.orchestra.run.smithi049.stdout:950 punch from 0x37abbc2 to 0x37af946, (0x3d84 bytes) 2024-04-24T18:07:50.121 INFO:teuthology.orchestra.run.smithi049.stdout:951 read 0x24cb9ff thru 0x24cd1cc (0x17ce bytes) 2024-04-24T18:07:50.121 INFO:teuthology.orchestra.run.smithi049.stdout:952 writesame 0x9f52dc5 thru 0x9f61820 data_size 0x35(0xea5c bytes) 2024-04-24T18:07:50.123 INFO:teuthology.orchestra.run.smithi049.stdout:953 read 0x29117cd thru 0x2911bca (0x3fe bytes) 2024-04-24T18:07:50.124 INFO:teuthology.orchestra.run.smithi049.stdout:954 read 0x6b23005 thru 0x6b2d595 (0xa591 bytes) 2024-04-24T18:07:50.124 INFO:teuthology.orchestra.run.smithi049.stdout:955 write 0x5568d1 thru 0x557eae (0x15de bytes) 2024-04-24T18:07:50.150 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:50.150 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:50.161 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:07:50.188 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:50.196+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:50.200 INFO:teuthology.orchestra.run.smithi049.stdout:956 trunc from 0xb5ff455 to 0x2e941ba 2024-04-24T18:07:50.289 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:50.296+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:50.302 INFO:teuthology.orchestra.run.smithi049.stdout:957 write 0x38556d8 thru 0x38606a2 (0xafcb bytes) 2024-04-24T18:07:50.312 INFO:teuthology.orchestra.run.smithi049.stdout:959 write 0x14fcbf4 thru 0x1501722 (0x4b2f bytes) 2024-04-24T18:07:50.362 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:50.362 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:50.373 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:07:50.415 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:50.423+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:50.437 INFO:teuthology.orchestra.run.smithi049.stdout:960 trunc from 0x38606a3 to 0x8aabd7e 2024-04-24T18:07:50.443 INFO:teuthology.orchestra.run.smithi049.stdout:961 trunc from 0x8aabd7e to 0x6f1fab2 2024-04-24T18:07:50.461 INFO:teuthology.orchestra.run.smithi049.stdout:962 write 0xe6a1be8 thru 0xe6a391c (0x1d35 bytes) 2024-04-24T18:07:50.472 INFO:teuthology.orchestra.run.smithi049.stdout:963 write 0x795f63f thru 0x796128e (0x1c50 bytes) 2024-04-24T18:07:50.482 INFO:teuthology.orchestra.run.smithi049.stdout:964 write 0x9ac1d63 thru 0x9ac4645 (0x28e3 bytes) 2024-04-24T18:07:50.488 INFO:teuthology.orchestra.run.smithi049.stdout:965 write 0x32b7c43 thru 0x32c0aef (0x8ead bytes) 2024-04-24T18:07:50.491 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:50.499+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:50.504 INFO:teuthology.orchestra.run.smithi049.stdout:967 trunc from 0xe6a391d to 0x5fa9765 2024-04-24T18:07:50.516 INFO:teuthology.orchestra.run.smithi049.stdout:968 read 0x575cb6e thru 0x5765e55 (0x92e8 bytes) 2024-04-24T18:07:50.516 INFO:teuthology.orchestra.run.smithi049.stdout:969 punch from 0x57cf9d5 to 0x57d112f, (0x175a bytes) 2024-04-24T18:07:50.529 INFO:teuthology.orchestra.run.smithi049.stdout:970 write 0xcc9eb60 thru 0xcca6931 (0x7dd2 bytes) 2024-04-24T18:07:50.536 INFO:teuthology.orchestra.run.smithi049.stdout:971 trunc from 0xcca6932 to 0xc4201a5 2024-04-24T18:07:50.548 INFO:teuthology.orchestra.run.smithi049.stdout:972 punch from 0x29ebd11 to 0x29f299f, (0x6c8e bytes) 2024-04-24T18:07:50.550 INFO:teuthology.orchestra.run.smithi049.stdout:973 trunc from 0xc4201a5 to 0xdc9b048 2024-04-24T18:07:50.558 INFO:teuthology.orchestra.run.smithi049.stdout:974 write 0x1e47727 thru 0x1e4fdef (0x86c9 bytes) 2024-04-24T18:07:50.567 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:50.567 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:50.581 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:07:50.584 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.4 to 1 (10000) 2024-04-24T18:07:50.589 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:07:50.592 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:50.599+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:50.597 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-24T18:07:50.685 INFO:teuthology.orchestra.run.smithi049.stdout:975 read 0xc089bbe thru 0xc08c3d5 (0x2818 bytes) 2024-04-24T18:07:50.686 INFO:teuthology.orchestra.run.smithi049.stdout:976 punch from 0xadb88cf to 0xadbd3d7, (0x4b08 bytes) 2024-04-24T18:07:50.686 INFO:teuthology.orchestra.run.smithi049.stdout:977 read 0xb414c6a thru 0xb4163bb (0x1752 bytes) 2024-04-24T18:07:50.686 INFO:teuthology.orchestra.run.smithi049.stdout:978 write 0xc84b32d thru 0xc850327 (0x4ffb bytes) 2024-04-24T18:07:50.692 INFO:teuthology.orchestra.run.smithi049.stdout:979 write 0xf41c4c thru 0xf4d53b (0xb8f0 bytes) 2024-04-24T18:07:50.692 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:50.700+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:50.742 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:50.742 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:50.755 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:07:50.799 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:50.808+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:50.828 INFO:teuthology.orchestra.run.smithi049.stdout:981 write 0x990e653 thru 0x9915297 (0x6c45 bytes) 2024-04-24T18:07:50.837 INFO:teuthology.orchestra.run.smithi049.stdout:982 write 0x12c2df3 thru 0x12cf63d (0xc84b bytes) 2024-04-24T18:07:50.894 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:50.902+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:50.897 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:50.897 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:50.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:07:50.996 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:51.004+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:51.039 INFO:teuthology.orchestra.run.smithi049.stdout:983 writesame 0x21b8062 thru 0x21bd9a4 data_size 0x3(0x5943 bytes) 2024-04-24T18:07:51.075 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:51.075 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:51.085 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:07:51.096 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:51.104+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:51.197 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:51.205+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:51.251 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:51.251 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:51.264 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:07:51.298 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:51.306+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:51.360 INFO:teuthology.orchestra.run.smithi049.stdout:986 write 0xaf24046 thru 0xaf2b139 (0x70f4 bytes) 2024-04-24T18:07:51.370 INFO:teuthology.orchestra.run.smithi049.stdout:991 read 0xc9b0aba thru 0xc9b328e (0x27d5 bytes) 2024-04-24T18:07:51.379 INFO:teuthology.orchestra.run.smithi049.stdout:992 trunc from 0xdc9b048 to 0x627a57e 2024-04-24T18:07:51.392 INFO:teuthology.orchestra.run.smithi049.stdout:993 punch from 0x33cde02 to 0x33d9b39, (0xbd37 bytes) 2024-04-24T18:07:51.392 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.392 INFO:teuthology.orchestra.run.smithi049.stdout:996 writesame 0x190587c thru 0x191223f data_size 0x4(0xc9c4 bytes) 2024-04-24T18:07:51.399 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:51.407+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:51.404 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:51.404 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:51.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:07:51.445 INFO:teuthology.orchestra.run.smithi049.stdout:997 write 0x60a36f4 thru 0x60a7ec6 (0x47d3 bytes) 2024-04-24T18:07:51.455 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.464 INFO:teuthology.orchestra.run.smithi049.stdout:999 writesame 0x8a16836 thru 0x8a217d0 data_size 0x51(0xaf9b bytes) 2024-04-24T18:07:51.464 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.465 INFO:teuthology.orchestra.run.smithi049.stdout:1002 punch from 0x65e9b78 to 0x65ec6ba, (0x2b42 bytes) 2024-04-24T18:07:51.470 INFO:teuthology.orchestra.run.smithi049.stdout:1003 write 0x8fae306 thru 0x8fb5659 (0x7354 bytes) 2024-04-24T18:07:51.477 INFO:teuthology.orchestra.run.smithi049.stdout:1004 punch from 0x2ca57b4 to 0x2cb4cda, (0xf526 bytes) 2024-04-24T18:07:51.477 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.477 INFO:teuthology.orchestra.run.smithi049.stdout:1008 read 0x73a318c thru 0x73b01df (0xd054 bytes) 2024-04-24T18:07:51.477 INFO:teuthology.orchestra.run.smithi049.stdout:1009 read 0x27e02d4 thru 0x27e0eba (0xbe7 bytes) 2024-04-24T18:07:51.486 INFO:teuthology.orchestra.run.smithi049.stdout:1010 write 0xc6a5b13 thru 0xc6acacf (0x6fbd bytes) 2024-04-24T18:07:51.494 INFO:teuthology.orchestra.run.smithi049.stdout:1011 punch from 0x43072ba to 0x430cc93, (0x59d9 bytes) 2024-04-24T18:07:51.494 INFO:teuthology.orchestra.run.smithi049.stdout:1012 read 0x884ac0f thru 0x8851d11 (0x7103 bytes) 2024-04-24T18:07:51.494 INFO:teuthology.orchestra.run.smithi049.stdout:1013 punch from 0x60f049a to 0x60f81ca, (0x7d30 bytes) 2024-04-24T18:07:51.494 INFO:teuthology.orchestra.run.smithi049.stdout:1014 write 0x538ce8 thru 0x5464ea (0xd803 bytes) 2024-04-24T18:07:51.499 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:51.507+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:51.525 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.525 INFO:teuthology.orchestra.run.smithi049.stdout:1016 writesame 0x2a5c5b9 thru 0x2a67ea1 data_size 0x1f(0xb8e9 bytes) 2024-04-24T18:07:51.533 INFO:teuthology.orchestra.run.smithi049.stdout:1017 trunc from 0xc6acad0 to 0x6abd66f 2024-04-24T18:07:51.556 INFO:teuthology.orchestra.run.smithi049.stdout:1018 write 0xc1e3945 thru 0xc1e7c7a (0x4336 bytes) 2024-04-24T18:07:51.564 INFO:teuthology.orchestra.run.smithi049.stdout:1021 read 0x96db51f thru 0x96dc8e7 (0x13c9 bytes) 2024-04-24T18:07:51.565 INFO:teuthology.orchestra.run.smithi049.stdout:1022 write 0x32957f2 thru 0x32a4540 (0xed4f bytes) 2024-04-24T18:07:51.567 INFO:teuthology.orchestra.run.smithi049.stdout:1024 read 0x7c893b5 thru 0x7c95cd4 (0xc920 bytes) 2024-04-24T18:07:51.567 INFO:teuthology.orchestra.run.smithi049.stdout:1025 read 0xc03a5b thru 0xc103c4 (0xc96a bytes) 2024-04-24T18:07:51.567 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.567 INFO:teuthology.orchestra.run.smithi049.stdout:1028 write 0xb25fa88 thru 0xb26bda8 (0xc321 bytes) 2024-04-24T18:07:51.578 INFO:teuthology.orchestra.run.smithi049.stdout:1032 punch from 0x77f3cbe to 0x7800a42, (0xcd84 bytes) 2024-04-24T18:07:51.578 INFO:teuthology.orchestra.run.smithi049.stdout:1033 write 0x9157205 thru 0x9160747 (0x9543 bytes) 2024-04-24T18:07:51.587 INFO:teuthology.orchestra.run.smithi049.stdout:1034 read 0x80f8fa4 thru 0x80fda13 (0x4a70 bytes) 2024-04-24T18:07:51.588 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.588 INFO:teuthology.orchestra.run.smithi049.stdout:1036 read 0x4567ec3 thru 0x456943a (0x1578 bytes) 2024-04-24T18:07:51.588 INFO:teuthology.orchestra.run.smithi049.stdout:1037 write 0xa523e3c thru 0xa527395 (0x355a bytes) 2024-04-24T18:07:51.596 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:07:51.597 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.5 to 1 (10000) 2024-04-24T18:07:51.598 INFO:teuthology.orchestra.run.smithi049.stdout:1038 writesame 0x3127868 thru 0x313331d data_size 0x1(0xbab6 bytes) 2024-04-24T18:07:51.600 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:51.600 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:51.600 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:51.608+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:51.613 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:07:51.614 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.614 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.614 INFO:teuthology.orchestra.run.smithi049.stdout:1042 write 0x17d02b5 thru 0x17de723 (0xe46f bytes) 2024-04-24T18:07:51.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-24T18:07:51.617 INFO:teuthology.orchestra.run.smithi049.stdout:1045 write 0xa97b86d thru 0xa984517 (0x8cab bytes) 2024-04-24T18:07:51.628 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.639 INFO:teuthology.orchestra.run.smithi049.stdout:1047 trunc from 0xc1e7c7b to 0x23e288d 2024-04-24T18:07:51.685 INFO:teuthology.orchestra.run.smithi049.stdout:1048 write 0xebc8cde thru 0xebd343d (0xa760 bytes) 2024-04-24T18:07:51.692 INFO:teuthology.orchestra.run.smithi049.stdout:1049 punch from 0x9f9a12 to 0x9fd37c, (0x396a bytes) 2024-04-24T18:07:51.693 INFO:teuthology.orchestra.run.smithi049.stdout:1050 writesame 0xde65b22 thru 0xde718aa data_size 0xb(0xbd89 bytes) 2024-04-24T18:07:51.699 INFO:teuthology.orchestra.run.smithi049.stdout:1051 punch from 0x4d95471 to 0x4d9bcdc, (0x686b bytes) 2024-04-24T18:07:51.699 INFO:teuthology.orchestra.run.smithi049.stdout:1052 write 0x6b73bdc thru 0x6b801b6 (0xc5db bytes) 2024-04-24T18:07:51.701 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:51.708+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:51.706 INFO:teuthology.orchestra.run.smithi049.stdout:1053 read 0xb2cabc4 thru 0xb2d9af4 (0xef31 bytes) 2024-04-24T18:07:51.706 INFO:teuthology.orchestra.run.smithi049.stdout:1054 writesame 0x94ba6f2 thru 0x94bd76b data_size 0x22(0x307a bytes) 2024-04-24T18:07:51.728 INFO:teuthology.orchestra.run.smithi049.stdout:1055 trunc from 0xebd343e to 0x1e36ac7 2024-04-24T18:07:51.756 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:51.756 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:51.767 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:07:51.768 INFO:teuthology.orchestra.run.smithi049.stdout:1058 trunc from 0x1e36ac7 to 0x9a97bbd 2024-04-24T18:07:51.773 INFO:teuthology.orchestra.run.smithi049.stdout:1059 read 0x35a8305 thru 0x35a87c3 (0x4bf bytes) 2024-04-24T18:07:51.773 INFO:teuthology.orchestra.run.smithi049.stdout:1060 writesame 0x2150f44 thru 0x2151772 data_size 0x1(0x82f bytes) 2024-04-24T18:07:51.777 INFO:teuthology.orchestra.run.smithi049.stdout:1061 punch from 0x8b8149 to 0x8b836c, (0x223 bytes) 2024-04-24T18:07:51.777 INFO:teuthology.orchestra.run.smithi049.stdout:1062 writesame 0x2b48512 thru 0x2b49435 data_size 0x6(0xf24 bytes) 2024-04-24T18:07:51.786 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.786 INFO:teuthology.orchestra.run.smithi049.stdout:1066 read 0x45443de thru 0x4548dcb (0x49ee bytes) 2024-04-24T18:07:51.786 INFO:teuthology.orchestra.run.smithi049.stdout:1067 read 0x39a5915 thru 0x39aeaa9 (0x9195 bytes) 2024-04-24T18:07:51.786 INFO:teuthology.orchestra.run.smithi049.stdout:1068 write 0x887dfde thru 0x888d46a (0xf48d bytes) 2024-04-24T18:07:51.800 INFO:teuthology.orchestra.run.smithi049.stdout:1069 read 0x753e038 thru 0x754b4db (0xd4a4 bytes) 2024-04-24T18:07:51.801 INFO:teuthology.orchestra.run.smithi049.stdout:1070 writesame 0x26e24e4 thru 0x26ecd4d data_size 0x1(0xa86a bytes) 2024-04-24T18:07:51.802 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:51.809+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:51.814 INFO:teuthology.orchestra.run.smithi049.stdout:1071 read 0x65cdeab thru 0x65d4e1a (0x6f70 bytes) 2024-04-24T18:07:51.814 INFO:teuthology.orchestra.run.smithi049.stdout:1072 read 0x4dd44f1 thru 0x4dde2c4 (0x9dd4 bytes) 2024-04-24T18:07:51.814 INFO:teuthology.orchestra.run.smithi049.stdout:1073 read 0x4015448 thru 0x40248be (0xf477 bytes) 2024-04-24T18:07:51.814 INFO:teuthology.orchestra.run.smithi049.stdout:1074 write 0x3252c8a thru 0x3259e9f (0x7216 bytes) 2024-04-24T18:07:51.817 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:07:51.827 INFO:teuthology.orchestra.run.smithi049.stdout:1075 read 0x901875f thru 0x901c1cd (0x3a6f bytes) 2024-04-24T18:07:51.827 INFO:teuthology.orchestra.run.smithi049.stdout:1076 writesame 0x21b5493 thru 0x21b99aa data_size 0x58(0x4518 bytes) 2024-04-24T18:07:51.835 INFO:teuthology.orchestra.run.smithi049.stdout:1077 write 0x1ab187f thru 0x1abbe06 (0xa588 bytes) 2024-04-24T18:07:51.839 INFO:teuthology.orchestra.run.smithi049.stdout:1078 punch from 0x758bf24 to 0x7597217, (0xb2f3 bytes) 2024-04-24T18:07:51.839 INFO:teuthology.orchestra.run.smithi049.stdout:1079 read 0x4816e30 thru 0x481dc7d (0x6e4e bytes) 2024-04-24T18:07:51.840 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:51.847 INFO:teuthology.orchestra.run.smithi049.stdout:1081 trunc from 0x9a97bbd to 0x358ff3f 2024-04-24T18:07:51.857 INFO:teuthology.orchestra.run.smithi049.stdout:1082 read 0x1ca2a49 thru 0x1ca7a19 (0x4fd1 bytes) 2024-04-24T18:07:51.858 INFO:teuthology.orchestra.run.smithi049.stdout:1083 write 0xf84f54 thru 0xf89be7 (0x4c94 bytes) 2024-04-24T18:07:51.860 INFO:teuthology.orchestra.run.smithi049.stdout:1085 read 0x2d91f84 thru 0x2d9a32a (0x83a7 bytes) 2024-04-24T18:07:51.877 INFO:teuthology.orchestra.run.smithi049.stdout:1086 write 0xd275491 thru 0xd275bf7 (0x767 bytes) 2024-04-24T18:07:51.881 INFO:teuthology.orchestra.run.smithi049.stdout:1087 read 0x2de643e thru 0x2df076f (0xa332 bytes) 2024-04-24T18:07:51.881 INFO:teuthology.orchestra.run.smithi049.stdout:1088 read 0x635807a thru 0x63641d2 (0xc159 bytes) 2024-04-24T18:07:51.881 INFO:teuthology.orchestra.run.smithi049.stdout:1090 write 0xbec7119 thru 0xbed56e7 (0xe5cf bytes) 2024-04-24T18:07:51.888 INFO:teuthology.orchestra.run.smithi049.stdout:1091 writesame 0xa67f056 thru 0xa68de89 data_size 0xa(0xee34 bytes) 2024-04-24T18:07:51.901 INFO:teuthology.orchestra.run.smithi049.stdout:1092 write 0xd532f39 thru 0xd537dd4 (0x4e9c bytes) 2024-04-24T18:07:51.903 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:51.911+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:51.908 INFO:teuthology.orchestra.run.smithi049.stdout:1094 writesame 0x435d0e thru 0x443016 data_size 0x5(0xd309 bytes) 2024-04-24T18:07:51.909 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:51.909 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:51.913 INFO:teuthology.orchestra.run.smithi049.stdout:1095 read 0x9f3b05f thru 0x9f423b7 (0x7359 bytes) 2024-04-24T18:07:51.913 INFO:teuthology.orchestra.run.smithi049.stdout:1096 read 0x586f66e thru 0x5873aae (0x4441 bytes) 2024-04-24T18:07:51.913 INFO:teuthology.orchestra.run.smithi049.stdout:1097 clone 7 order 20 su 32768 sc 14 2024-04-24T18:07:51.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:07:51.979 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:51.999 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:52.003 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:52.011+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:52.018 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:52.036 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:52.044 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:52.044 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:52.054 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:07:52.055 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:52.074 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:52.093 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:52.104 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:52.113+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:52.111 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:52.113 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:52.180 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:52.181 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:52.181 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:52.181 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:52.190 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:07:52.232 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:52.240+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:52.306 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:52.314+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:52.355 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:52.355 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:52.355 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:52.355 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:52.355 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:52.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:07:52.408 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:52.417+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:52.491 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:52.491 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881998359000001, 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.492 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:52.497 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881929307, 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:07:52.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:07:52.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:07:52.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:07:52.502 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881904892000001, 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:07:52.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.508 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881881743000001, 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:07:52.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:07:52.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881801175, 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.514 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:52.519 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881776365, 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:07:52.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:07:52.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881756252999999, 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.530 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.88168559, 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.881673816999999, 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:52.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:07:52.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:132 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.461690454999999, 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0070057069999999999, 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 132 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036534+0000", 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036533+0000", 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036535+0000", 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036539+0000", 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3459999999999998e-06 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036541+0000", 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9520000000000001e-06 2024-04-24T18:07:52.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037943+0000", 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001402401 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038021+0000", 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8071000000000005e-05 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.038150+0000", 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012869199999999999 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.039054+0000", 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00090387400000000004 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043447+0000", 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043929110000000002 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043497+0000", 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0420999999999997e-05 2024-04-24T18:07:52.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043539+0000", 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2255000000000001e-05 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:136 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.461388204999999, 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0068998769999999996, 2024-04-24T18:07:52.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036836+0000", 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036839+0000", 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9739999999999998e-06 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036844+0000", 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3079999999999998e-06 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036846+0000", 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4099999999999998e-06 2024-04-24T18:07:52.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040072+0000", 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003225805 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040118+0000", 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6026999999999998e-05 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040191+0000", 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2983000000000002e-05 2024-04-24T18:07:52.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042013+0000", 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001822118 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043702+0000", 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0016890329999999999 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043721+0000", 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8754999999999999e-05 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043736+0000", 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4725e-05 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.461351988000001, 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.553 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:07:52.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:07:52.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:07:52.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.558 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.461304806000001, 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:52.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:07:52.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:07:52.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:07:52.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:07:52.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.461279685000001, 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.461227551, 2024-04-24T18:07:52.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:07:52.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:07:52.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:07:52.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:07:52.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.461164087, 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:07:52.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 30.949993788, 2024-04-24T18:07:52.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073643390000000001, 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 156 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548239+0000", 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8899999999999996e-06 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548246+0000", 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9370000000000004e-06 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548259+0000", 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549586+0000", 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001326743 2024-04-24T18:07:52.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549641+0000", 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5856999999999997e-05 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549713+0000", 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1970000000000004e-05 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550899+0000", 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001185567 2024-04-24T18:07:52.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555532+0000", 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0046325760000000002 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555573+0000", 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1730000000000002e-05 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555595+0000", 2024-04-24T18:07:52.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1324000000000001e-05 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 30.949958415000001, 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.586 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.591 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 30.949933388000002, 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:07:52.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 30.949854151, 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:52.597 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:52.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.600 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.600 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:07:52.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:07:52.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:07:52.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:07:52.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:52.604 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:52.605 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.6 to 1 (10000) 2024-04-24T18:07:52.606 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:52.613+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:52.606 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:07:52.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-24T18:07:52.656 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone6 from 0xd537dd5 (overlap 0x1e36ac7) to 0x0 2024-04-24T18:07:52.715 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:52.723+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33452 4.1e 4:7cdd4edb:::rbd_data.117432c990a0.0000000000000031:head [stat out=16b,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.806301+0000", 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0047936749999999998, 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048235600000000002, 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "waiting for sub ops", 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:52.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33452 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806301+0000", 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806301+0000", 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806300+0000", 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806309+0000", 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8109999999999997e-06 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806332+0000", 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2370000000000001e-05 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806341+0000", 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3570000000000004e-06 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806399+0000", 2024-04-24T18:07:52.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7939999999999998e-05 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806497+0000", 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7578999999999995e-05 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.806696+0000", 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000199669 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.808259+0000", 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001562962 2024-04-24T18:07:52.807 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33453 4.3 4:c09fd005:::rbd_data.117432c990a0.0000000000000032:head [stat out=16b,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.807847+0000", 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0032479789999999998, 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003355743, 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "waiting for sub ops", 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33453 2024-04-24T18:07:52.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807847+0000", 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807847+0000", 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807846+0000", 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:52.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807852+0000", 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807868+0000", 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5744e-05 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807875+0000", 2024-04-24T18:07:52.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1450000000000002e-06 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807917+0000", 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2701999999999999e-05 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.807997+0000", 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9473000000000003e-05 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.808191+0000", 2024-04-24T18:07:52.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019443999999999999 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.809058+0000", 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00086656400000000001 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.812 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33455 4.1e 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head [stat,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.811186+0000", 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9999094, 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000108589, 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33455 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.811186+0000", 2024-04-24T18:07:52.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.811186+0000", 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.811185+0000", 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.811190+0000", 2024-04-24T18:07:52.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8350000000000003e-06 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.811201+0000", 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1022000000000001e-05 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.811206+0000", 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7679999999999999e-06 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.811246+0000", 2024-04-24T18:07:52.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0762e-05 2024-04-24T18:07:52.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:52.816 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:52.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:52.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:52.816 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:07:52.816 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 3 2024-04-24T18:07:52.816 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:52.817 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:52.825+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:52.818 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:07:52.916 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #5, image image_client.0-clone5 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent6 2024-04-24T18:07:52.918 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:52.926+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:52.990 INFO:teuthology.orchestra.run.smithi049.stdout:1098 read 0xb0db007 thru 0xb0e28d2 (0x78cc bytes) 2024-04-24T18:07:52.991 INFO:teuthology.orchestra.run.smithi049.stdout:1100 write 0x5917319 thru 0x5925b8f (0xe877 bytes) 2024-04-24T18:07:53.006 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:53.006 INFO:teuthology.orchestra.run.smithi049.stdout:1102 write 0x9e8920f thru 0x9e93d5d (0xab4f bytes) 2024-04-24T18:07:53.015 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:53.015 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:53.016 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:53.016 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:53.016 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:53.019 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:53.028+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:53.030 INFO:teuthology.orchestra.run.smithi049.stdout:1103 trunc from 0xd537dd5 to 0xb7cfea 2024-04-24T18:07:53.034 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:07:53.122 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:53.130+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:53.221 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:53.230+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:53.261 INFO:teuthology.orchestra.run.smithi049.stdout:1104 trunc from 0xb7cfea to 0xaff5483 2024-04-24T18:07:53.267 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:53.267 INFO:teuthology.orchestra.run.smithi049.stdout:1106 write 0x4e78afd thru 0x4e8620a (0xd70e bytes) 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31810 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 2920448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.586465032, 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052993206000000001, 2024-04-24T18:07:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31810 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:07:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691739+0000", 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691742+0000", 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2800000000000002e-06 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691747+0000", 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4319999999999998e-06 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691750+0000", 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.247e-06 2024-04-24T18:07:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744520+0000", 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052769498999999997 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744623+0000", 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000102808 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744733+0000", 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110282 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.57288027200000002, 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:07:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:07:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:07:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32297 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3233792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:07:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.56050618900000004, 2024-04-24T18:07:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052981070999999998, 2024-04-24T18:07:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32297 2024-04-24T18:07:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:07:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:07:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717698+0000", 2024-04-24T18:07:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717705+0000", 2024-04-24T18:07:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5529999999999998e-06 2024-04-24T18:07:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717711+0000", 2024-04-24T18:07:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3849999999999999e-06 2024-04-24T18:07:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717715+0000", 2024-04-24T18:07:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:07:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770506+0000", 2024-04-24T18:07:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052791267000000003 2024-04-24T18:07:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770587+0000", 2024-04-24T18:07:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0783000000000002e-05 2024-04-24T18:07:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770680+0000", 2024-04-24T18:07:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2497000000000006e-05 2024-04-24T18:07:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32299 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:07:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.56048518599999997, 2024-04-24T18:07:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053161606, 2024-04-24T18:07:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32299 2024-04-24T18:07:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:07:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:07:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717719+0000", 2024-04-24T18:07:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717722+0000", 2024-04-24T18:07:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.43e-06 2024-04-24T18:07:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717729+0000", 2024-04-24T18:07:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.576e-06 2024-04-24T18:07:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717733+0000", 2024-04-24T18:07:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0119999999999999e-06 2024-04-24T18:07:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770707+0000", 2024-04-24T18:07:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052974083999999998 2024-04-24T18:07:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770791+0000", 2024-04-24T18:07:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3418000000000007e-05 2024-04-24T18:07:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770881+0000", 2024-04-24T18:07:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0490999999999998e-05 2024-04-24T18:07:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32301 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:07:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.56046731599999999, 2024-04-24T18:07:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053336189999999999, 2024-04-24T18:07:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32301 2024-04-24T18:07:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:07:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:07:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717737+0000", 2024-04-24T18:07:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717740+0000", 2024-04-24T18:07:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4090000000000001e-06 2024-04-24T18:07:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717746+0000", 2024-04-24T18:07:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9989999999999997e-06 2024-04-24T18:07:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717752+0000", 2024-04-24T18:07:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0349999999999998e-06 2024-04-24T18:07:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770915+0000", 2024-04-24T18:07:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053162938 2024-04-24T18:07:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770989+0000", 2024-04-24T18:07:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4567999999999995e-05 2024-04-24T18:07:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771074+0000", 2024-04-24T18:07:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4598000000000006e-05 2024-04-24T18:07:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32303 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3270656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.56035423799999995, 2024-04-24T18:07:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053426976000000001, 2024-04-24T18:07:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32303 2024-04-24T18:07:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:07:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:07:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717850+0000", 2024-04-24T18:07:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717854+0000", 2024-04-24T18:07:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0509999999999998e-06 2024-04-24T18:07:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717863+0000", 2024-04-24T18:07:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.346e-06 2024-04-24T18:07:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717877+0000", 2024-04-24T18:07:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3559e-05 2024-04-24T18:07:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771102+0000", 2024-04-24T18:07:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053224869000000001 2024-04-24T18:07:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771184+0000", 2024-04-24T18:07:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2564000000000004e-05 2024-04-24T18:07:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771278+0000", 2024-04-24T18:07:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3178999999999997e-05 2024-04-24T18:07:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32305 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3278848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:07:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.56032207000000001, 2024-04-24T18:07:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053587411000000001, 2024-04-24T18:07:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32305 2024-04-24T18:07:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:07:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:07:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717882+0000", 2024-04-24T18:07:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717885+0000", 2024-04-24T18:07:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8679999999999999e-06 2024-04-24T18:07:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717892+0000", 2024-04-24T18:07:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:07:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717896+0000", 2024-04-24T18:07:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9360000000000001e-06 2024-04-24T18:07:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771305+0000", 2024-04-24T18:07:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053409310000000002 2024-04-24T18:07:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771373+0000", 2024-04-24T18:07:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8417999999999994e-05 2024-04-24T18:07:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771470+0000", 2024-04-24T18:07:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6827e-05 2024-04-24T18:07:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32307 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3289088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.56030502900000001, 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053767057, 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32307 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:07:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717899+0000", 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717902+0000", 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5019999999999999e-06 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717908+0000", 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7699999999999998e-06 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717911+0000", 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4979999999999998e-06 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771498+0000", 2024-04-24T18:07:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053586747999999997 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771571+0000", 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3168000000000001e-05 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771667+0000", 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5810000000000006e-05 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32311 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3307520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55900864400000005, 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053356705999999997, 2024-04-24T18:07:53.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32311 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719199+0000", 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.88e-06 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719205+0000", 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719209+0000", 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0749999999999999e-06 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772008+0000", 2024-04-24T18:07:53.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052798572000000002 2024-04-24T18:07:53.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772048+0000", 2024-04-24T18:07:53.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0238999999999999e-05 2024-04-24T18:07:53.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772553+0000", 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050474099999999996 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32313 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3315712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55899058599999996, 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053105309000000003, 2024-04-24T18:07:53.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32313 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719216+0000", 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719223+0000", 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3010000000000003e-06 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719226+0000", 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7610000000000001e-06 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772130+0000", 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052903238999999998 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772209+0000", 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9486000000000004e-05 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772320+0000", 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011052099999999999 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32315 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3334144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55897394300000003, 2024-04-24T18:07:53.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053250991999999997, 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32315 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719233+0000", 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1789999999999998e-06 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719240+0000", 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5689999999999999e-06 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719244+0000", 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5870000000000001e-06 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772345+0000", 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053100972000000003 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772413+0000", 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8071000000000006e-05 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772482+0000", 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8950999999999998e-05 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32317 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55889188199999995, 2024-04-24T18:07:53.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053370316000000001, 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32317 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719312+0000", 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719316+0000", 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9400000000000004e-06 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719326+0000", 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2439999999999999e-06 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719331+0000", 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7049999999999996e-06 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772583+0000", 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053251860999999998 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772628+0000", 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5289e-05 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772683+0000", 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4882000000000001e-05 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32319 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3352576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.355 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.558868539, 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053459060000000003, 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32319 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719338+0000", 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5490000000000001e-06 2024-04-24T18:07:53.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719346+0000", 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5059999999999998e-06 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719350+0000", 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1999999999999996e-06 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772709+0000", 2024-04-24T18:07:53.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053359182999999998 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772748+0000", 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8654e-05 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772795+0000", 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7437e-05 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.359 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32321 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3360768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55885032899999998, 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053627222000000002, 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32321 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:07:53.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719354+0000", 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719361+0000", 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7449999999999998e-06 2024-04-24T18:07:53.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719367+0000", 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3940000000000001e-06 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719378+0000", 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1204e-05 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772830+0000", 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053451920999999999 2024-04-24T18:07:53.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772896+0000", 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5623000000000005e-05 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772982+0000", 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5729000000000005e-05 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32325 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3379200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.363 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720354+0000", 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55785121299999996, 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052932845999999999, 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32325 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720354+0000", 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720354+0000", 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720353+0000", 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720356+0000", 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.824e-06 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720362+0000", 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1670000000000004e-06 2024-04-24T18:07:53.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720366+0000", 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9920000000000001e-06 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773163+0000", 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052797139999999999 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773223+0000", 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9380999999999999e-05 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773286+0000", 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3813999999999996e-05 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32327 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3389440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55783412099999996, 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053038750000000003, 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32327 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720370+0000", 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720373+0000", 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5830000000000001e-06 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720380+0000", 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1550000000000001e-06 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720384+0000", 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0260000000000001e-06 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773322+0000", 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052937495000000001 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773364+0000", 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2561999999999997e-05 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773409+0000", 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5333000000000001e-05 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32329 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3397632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.371 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55781642200000003, 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053169842000000002, 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32329 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720391+0000", 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7779999999999999e-06 2024-04-24T18:07:53.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720396+0000", 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6729999999999997e-06 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720400+0000", 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7179999999999998e-06 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773435+0000", 2024-04-24T18:07:53.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053035169 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773497+0000", 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1715000000000001e-05 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773558+0000", 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1171000000000001e-05 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32331 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3416064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.557800293, 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053284320000000003, 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32331 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720404+0000", 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720406+0000", 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3190000000000002e-06 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720412+0000", 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8359999999999997e-06 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720416+0000", 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000004e-06 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773583+0000", 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053167076000000001 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773638+0000", 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105999999999997e-05 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773689+0000", 2024-04-24T18:07:53.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0521e-05 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32333 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3424256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55778268099999995, 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053395479000000003, 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32333 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720424+0000", 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720430+0000", 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:07:53.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720434+0000", 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8210000000000003e-06 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773721+0000", 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053287297999999997 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773762+0000", 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1245000000000002e-05 2024-04-24T18:07:53.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773818+0000", 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5585999999999999e-05 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32388 4.3 4:c61fab1f:::rbd_object_map.117432c990a0.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.55568045200000005, 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.055543581000000002, 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32388 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722527+0000", 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0319999999999999e-06 2024-04-24T18:07:53.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722535+0000", 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9170000000000006e-06 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722539+0000", 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9709999999999998e-06 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774649+0000", 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052110109000000002 2024-04-24T18:07:53.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774719+0000", 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9562999999999998e-05 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775050+0000", 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000331078 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777149+0000", 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020996639999999998 2024-04-24T18:07:53.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777996+0000", 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084650899999999996 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778027+0000", 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0818999999999997e-05 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778068+0000", 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1338000000000003e-05 2024-04-24T18:07:53.389 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.390 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.390 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.390 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.390 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.390 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:53.391 INFO:teuthology.orchestra.run.smithi049.stdout:1107 clone 8 order 25 su 2097152 sc 11 2024-04-24T18:07:53.391 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:07:53.392 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:53.399+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:53.492 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:53.500+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:53.562 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:53.562 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:53.562 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:53.562 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:53.572 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:07:53.593 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:53.601+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:53.603 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.7 to 1 (10000) 2024-04-24T18:07:53.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:07:53.617 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-24T18:07:53.618 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:07:53.682 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone7 from 0xaff5483 (overlap 0xb7cfea) to 0x80ab23 2024-04-24T18:07:53.723 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:53.731+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:53.758 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:53.758 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:53.758 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:53.758 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:53.758 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:53.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:07:53.776 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #6, image image_client.0-clone6 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent7 2024-04-24T18:07:53.778 INFO:teuthology.orchestra.run.smithi049.stdout:1108 read 0xa2045b9 thru 0xa20b0a9 (0x6af1 bytes) 2024-04-24T18:07:53.778 INFO:teuthology.orchestra.run.smithi049.stdout:1109 trunc from 0xaff5483 to 0xa3a4a95 2024-04-24T18:07:53.794 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:53.802+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:53.807 INFO:teuthology.orchestra.run.smithi049.stdout:1110 writesame 0x244e61a thru 0x245078a data_size 0x1(0x2171 bytes) 2024-04-24T18:07:53.812 INFO:teuthology.orchestra.run.smithi049.stdout:1111 writesame 0x3a9c809 thru 0x3aa04df data_size 0x23(0x3cd7 bytes) 2024-04-24T18:07:53.814 INFO:teuthology.orchestra.run.smithi049.stdout:1112 read 0x883ad74 thru 0x88490a4 (0xe331 bytes) 2024-04-24T18:07:53.823 INFO:teuthology.orchestra.run.smithi049.stdout:1115 trunc from 0xa3a4a95 to 0x5e68dc9 2024-04-24T18:07:53.865 INFO:teuthology.orchestra.run.smithi049.stdout:1116 punch from 0x3c9f55f to 0x3caf3b8, (0xfe59 bytes) 2024-04-24T18:07:53.865 INFO:teuthology.orchestra.run.smithi049.stdout:1117 punch from 0x209fc7c to 0x20a2c31, (0x2fb5 bytes) 2024-04-24T18:07:53.865 INFO:teuthology.orchestra.run.smithi049.stdout:1118 read 0x42afe35 thru 0x42b0e22 (0xfee bytes) 2024-04-24T18:07:53.866 INFO:teuthology.orchestra.run.smithi049.stdout:1119 flatten 2024-04-24T18:07:53.870 INFO:teuthology.orchestra.run.smithi049.stdout:1120 read 0x280e7e5 thru 0x2819910 (0xb12c bytes) 2024-04-24T18:07:53.871 INFO:teuthology.orchestra.run.smithi049.stdout:1121 write 0x182ae18 thru 0x1832991 (0x7b7a bytes) 2024-04-24T18:07:53.882 INFO:teuthology.orchestra.run.smithi049.stdout:1122 write 0x9979b4c thru 0x997b0d5 (0x158a bytes) 2024-04-24T18:07:53.883 INFO:teuthology.orchestra.run.smithi049.stdout:1123 punch from 0x81bd3f to 0x8277d1, (0xba92 bytes) 2024-04-24T18:07:53.883 INFO:teuthology.orchestra.run.smithi049.stdout:1124 read 0x2b00d98 thru 0x2b0ca9c (0xbd05 bytes) 2024-04-24T18:07:53.883 INFO:teuthology.orchestra.run.smithi049.stdout:1125 read 0x581c040 thru 0x5824ae5 (0x8aa6 bytes) 2024-04-24T18:07:53.893 INFO:teuthology.orchestra.run.smithi049.stdout:1126 trunc from 0x997b0d6 to 0x1727f12 2024-04-24T18:07:53.895 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:53.903+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:53.900 INFO:teuthology.orchestra.run.smithi049.stdout:1127 read 0x156d09a thru 0x1574d33 (0x7c9a bytes) 2024-04-24T18:07:53.900 INFO:teuthology.orchestra.run.smithi049.stdout:1128 write 0xc56440 thru 0xc58a89 (0x264a bytes) 2024-04-24T18:07:53.902 INFO:teuthology.orchestra.run.smithi049.stdout:1129 read 0xf0329 thru 0xf2c02 (0x28da bytes) 2024-04-24T18:07:53.903 INFO:teuthology.orchestra.run.smithi049.stdout:1130 read 0x14ad906 thru 0x14bc940 (0xf03b bytes) 2024-04-24T18:07:53.910 INFO:teuthology.orchestra.run.smithi049.stdout:1132 writesame 0x30160a7 thru 0x3025640 data_size 0x3f(0xf59a bytes) 2024-04-24T18:07:53.911 INFO:teuthology.orchestra.run.smithi049.stdout:1133 clone 9 order 20 su 8192 sc 4 2024-04-24T18:07:53.958 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:20 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'10, mlcod=16'8)", 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.543681564000003, 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001123017, 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422302+0000", 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422414+0000", 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000112488 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422424+0000", 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0118e-05 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422431+0000", 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9319999999999999e-06 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422477+0000", 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5738999999999997e-05 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.422501+0000", 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4032e-05 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.423397+0000", 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089551500000000005 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.423425+0000", 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8775000000000001e-05 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:31 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'17, mlcod=16'14)", 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.533362087999997, 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001094629, 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432622+0000", 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432621+0000", 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432627+0000", 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5160000000000002e-06 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432635+0000", 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3669999999999997e-06 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432639+0000", 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9179999999999996e-06 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432678+0000", 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8627999999999997e-05 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.432701+0000", 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3246000000000001e-05 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433677+0000", 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097568000000000004 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433716+0000", 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9765999999999998e-05 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:32 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'18, mlcod=16'14)", 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.532811154000001, 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00093691599999999999, 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433173+0000", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433172+0000", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433240+0000", 2024-04-24T18:07:53.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7769999999999997e-05 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433253+0000", 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3118999999999999e-05 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433259+0000", 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3090000000000001e-06 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433287+0000", 2024-04-24T18:07:53.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7813e-05 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.433300+0000", 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3004e-05 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.434087+0000", 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00078750999999999995 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.434110+0000", 2024-04-24T18:07:53.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2198999999999999e-05 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:54 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'32, mlcod=16'29)", 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.517251995000002, 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0014283620000000001, 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448732+0000", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448731+0000", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448738+0000", 2024-04-24T18:07:53.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4590000000000003e-06 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448751+0000", 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2833000000000001e-05 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448757+0000", 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448800+0000", 2024-04-24T18:07:53.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3282999999999999e-05 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.448823+0000", 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2731999999999999e-05 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.450119+0000", 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001295942 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.450160+0000", 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1709999999999999e-05 2024-04-24T18:07:53.974 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:57 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'35, mlcod=16'33)", 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.513983230000001, 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00095936700000000003, 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:53.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452001+0000", 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452000+0000", 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452057+0000", 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7161999999999999e-05 2024-04-24T18:07:53.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452067+0000", 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0278999999999999e-05 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452072+0000", 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9429999999999999e-06 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452106+0000", 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4023000000000003e-05 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452128+0000", 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1145000000000002e-05 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452929+0000", 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00080152800000000003 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.452960+0000", 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0939000000000003e-05 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:68 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'42, mlcod=16'39)", 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.506725701999997, 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0009994750000000001, 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459258+0000", 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459262+0000", 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1609999999999996e-06 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459270+0000", 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5989999999999996e-06 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459274+0000", 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8539999999999996e-06 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459306+0000", 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1129e-05 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.459317+0000", 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1785e-05 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.460227+0000", 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090945999999999996 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.460258+0000", 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0899999999999999e-05 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:71 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'45, mlcod=16'43)", 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.504492026999998, 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00093247299999999996, 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461492+0000", 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461491+0000", 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461555+0000", 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3937999999999997e-05 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461565+0000", 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4800000000000007e-06 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461569+0000", 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5340000000000001e-06 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461616+0000", 2024-04-24T18:07:53.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6587e-05 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461640+0000", 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4130000000000001e-05 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462380+0000", 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074047800000000003 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462424+0000", 2024-04-24T18:07:53.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3980000000000002e-05 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:72 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'46, mlcod=16'43)", 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.504033964999998, 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010010850000000001, 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:53.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461950+0000", 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.461949+0000", 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462032+0000", 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2152999999999998e-05 2024-04-24T18:07:53.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462044+0000", 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2170999999999999e-05 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462051+0000", 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9600000000000003e-06 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462081+0000", 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9788e-05 2024-04-24T18:07:53.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462094+0000", 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3910999999999999e-05 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462904+0000", 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00081001000000000001 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.462951+0000", 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6597000000000001e-05 2024-04-24T18:07:53.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:83 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'53, mlcod=16'50)", 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.496603041, 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001099135, 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469381+0000", 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469380+0000", 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469387+0000", 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5220000000000004e-06 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469398+0000", 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1076e-05 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469405+0000", 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4699999999999996e-06 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469441+0000", 2024-04-24T18:07:53.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5429999999999998e-05 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469461+0000", 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9828999999999999e-05 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470443+0000", 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098290399999999994 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470480+0000", 2024-04-24T18:07:53.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6544000000000001e-05 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:84 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'54, mlcod=16'50)", 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.496067601999997, 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00094213999999999995, 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:53.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469916+0000", 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469980+0000", 2024-04-24T18:07:53.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4336000000000002e-05 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.469991+0000", 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1191e-05 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470001+0000", 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7680000000000003e-06 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470036+0000", 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5265000000000002e-05 2024-04-24T18:07:53.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470056+0000", 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9646000000000001e-05 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470828+0000", 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077209399999999997 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.470858+0000", 2024-04-24T18:07:53.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0639000000000003e-05 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:86 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'56, mlcod=16'52)", 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.494900608000002, 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010291390000000001, 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:53.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471083+0000", 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471089+0000", 2024-04-24T18:07:54.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3430000000000001e-06 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471104+0000", 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4617000000000001e-05 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471108+0000", 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6430000000000003e-06 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471168+0000", 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9917000000000002e-05 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.471187+0000", 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9147000000000002e-05 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.472073+0000", 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00088590000000000001 2024-04-24T18:07:54.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.472112+0000", 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999999e-05 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:89 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'59, mlcod=16'57)", 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.492108797, 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001052597, 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.003 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473875+0000", 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473874+0000", 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473941+0000", 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6638999999999997e-05 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473950+0000", 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.234e-06 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473955+0000", 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5179999999999998e-06 2024-04-24T18:07:54.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.473991+0000", 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5689000000000003e-05 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474013+0000", 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.209e-05 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474889+0000", 2024-04-24T18:07:54.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00087645799999999999 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.474928+0000", 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8591999999999999e-05 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:104 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'66, mlcod=16'64)", 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349762605999999, 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097261199999999998, 2024-04-24T18:07:54.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616221+0000", 2024-04-24T18:07:54.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616220+0000", 2024-04-24T18:07:54.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:54.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616228+0000", 2024-04-24T18:07:54.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1489999999999997e-06 2024-04-24T18:07:54.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616239+0000", 2024-04-24T18:07:54.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1029000000000001e-05 2024-04-24T18:07:54.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616244+0000", 2024-04-24T18:07:54.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0340000000000004e-06 2024-04-24T18:07:54.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616274+0000", 2024-04-24T18:07:54.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9957000000000001e-05 2024-04-24T18:07:54.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616290+0000", 2024-04-24T18:07:54.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.6035000000000001e-05 2024-04-24T18:07:54.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617130+0000", 2024-04-24T18:07:54.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084017 2024-04-24T18:07:54.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617194+0000", 2024-04-24T18:07:54.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4188000000000003e-05 2024-04-24T18:07:54.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.015 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.016 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:105 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'67, mlcod=16'64)", 2024-04-24T18:07:54.016 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:54.016 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349223320999997, 2024-04-24T18:07:54.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001106934, 2024-04-24T18:07:54.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.016 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:54.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616761+0000", 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616760+0000", 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616765+0000", 2024-04-24T18:07:54.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2839999999999997e-06 2024-04-24T18:07:54.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616775+0000", 2024-04-24T18:07:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7229999999999999e-06 2024-04-24T18:07:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616780+0000", 2024-04-24T18:07:54.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7620000000000003e-06 2024-04-24T18:07:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616818+0000", 2024-04-24T18:07:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8637999999999998e-05 2024-04-24T18:07:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.616839+0000", 2024-04-24T18:07:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0279e-05 2024-04-24T18:07:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617845+0000", 2024-04-24T18:07:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010063089999999999 2024-04-24T18:07:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617868+0000", 2024-04-24T18:07:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.249e-05 2024-04-24T18:07:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:106 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'68, mlcod=16'64)", 2024-04-24T18:07:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.348640238000002, 2024-04-24T18:07:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010242809999999999, 2024-04-24T18:07:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617344+0000", 2024-04-24T18:07:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617343+0000", 2024-04-24T18:07:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617408+0000", 2024-04-24T18:07:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4926999999999997e-05 2024-04-24T18:07:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617420+0000", 2024-04-24T18:07:54.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1977999999999999e-05 2024-04-24T18:07:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617423+0000", 2024-04-24T18:07:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.507e-06 2024-04-24T18:07:54.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617458+0000", 2024-04-24T18:07:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4999999999999997e-05 2024-04-24T18:07:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.617477+0000", 2024-04-24T18:07:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8902999999999998e-05 2024-04-24T18:07:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.618336+0000", 2024-04-24T18:07:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000859044 2024-04-24T18:07:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.618368+0000", 2024-04-24T18:07:54.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1656999999999999e-05 2024-04-24T18:07:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.035 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:109 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'71, mlcod=16'66)", 2024-04-24T18:07:54.035 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:54.035 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.346698046999997, 2024-04-24T18:07:54.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00099814200000000004, 2024-04-24T18:07:54.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.036 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:54.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619286+0000", 2024-04-24T18:07:54.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619285+0000", 2024-04-24T18:07:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619338+0000", 2024-04-24T18:07:54.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2568999999999998e-05 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619347+0000", 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2350000000000005e-06 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619355+0000", 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8199999999999997e-06 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619377+0000", 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2169999999999999e-05 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.619388+0000", 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0569e-05 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620251+0000", 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000863162 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620284+0000", 2024-04-24T18:07:54.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3145999999999997e-05 2024-04-24T18:07:54.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.042 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.043 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:112 2.0 e16/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'74, mlcod=16'68)", 2024-04-24T18:07:54.043 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:54.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.345100334000001, 2024-04-24T18:07:54.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00097267399999999998, 2024-04-24T18:07:54.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620884+0000", 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620883+0000", 2024-04-24T18:07:54.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620950+0000", 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7145999999999997e-05 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620958+0000", 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6240000000000002e-06 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620961+0000", 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3790000000000002e-06 2024-04-24T18:07:54.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.620992+0000", 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0784999999999997e-05 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621011+0000", 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8811999999999999e-05 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621819+0000", 2024-04-24T18:07:54.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00080833000000000001 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.621856+0000", 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7004e-05 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:118 2.0 e16/15 2:30306672:devicehealth::main.db.0000000000000000:head v 16'80, mlcod=16'78)", 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.337761493999999, 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00098543200000000006, 2024-04-24T18:07:54.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628222+0000", 2024-04-24T18:07:54.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628279+0000", 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6913000000000002e-05 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628289+0000", 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.8539999999999992e-06 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628293+0000", 2024-04-24T18:07:54.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4800000000000003e-06 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628329+0000", 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5945000000000002e-05 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.628349+0000", 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0367999999999999e-05 2024-04-24T18:07:54.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.629181+0000", 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00083184899999999996 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:20.629208+0000", 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6676e-05 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:154 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'108, mlcod=16'105)", 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.417189432000001, 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001000143, 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548794+0000", 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548799+0000", 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9579999999999998e-06 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548808+0000", 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7220000000000005e-06 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548813+0000", 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0690000000000001e-06 2024-04-24T18:07:54.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548846+0000", 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3419e-05 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.548857+0000", 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.094e-05 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549758+0000", 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00090098800000000003 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549795+0000", 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6531e-05 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4194.0:155 2.0 e17/15 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 17'109, mlcod=16'105)", 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 32.416792448000002, 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00096950400000000005, 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:54.056 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549191+0000", 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549275+0000", 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4623999999999995e-05 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549283+0000", 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590000000000002e-06 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549292+0000", 2024-04-24T18:07:54.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3340000000000008e-06 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549331+0000", 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8844999999999998e-05 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.549351+0000", 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9791000000000001e-05 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.550128+0000", 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077682700000000001 2024-04-24T18:07:54.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:21.550161+0000", 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3318999999999997e-05 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:54.060 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:54.062 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:07:54.062 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:54.069+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:54.067 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:54.069 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:07:54.081 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-04-24T18:07:53.606487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237806384,"num_objects":220,"num_object_clones":12,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":9,"num_read":37162,"num_read_kb":58020,"num_write":3576,"num_write_kb":42015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2706,"ondisk_log_size":2706,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2113908,"kb_used_data":11896,"kb_used_omap":11,"kb_used_meta":2101876,"kb_avail":658488972,"statfs":{"total":676457349120,"available":674292707328,"internally_reserved":0,"allocated":12181504,"data_stored":69422287,"data_compressed":1843193,"data_compressed_allocated":4894720,"data_compressed_original":64986882,"omap_allocated":11928,"internal_metadata":2152321384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":68018896,"num_objects":121,"num_object_clones":12,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":9,"num_read":28973,"num_read_kb":50112,"num_write":1622,"num_write_kb":31389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021738"},"pg_stats":[{"pgid":"4.1f","version":"41'64","reported_seq":935,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.657287+0000","last_change":"2024-04-24T18:07:38.288543+0000","last_active":"2024-04-24T18:07:51.657287+0000","last_peered":"2024-04-24T18:07:51.657287+0000","last_clean":"2024-04-24T18:07:51.657287+0000","last_became_active":"2024-04-24T18:07:36.550017+0000","last_became_peered":"2024-04-24T18:07:36.550017+0000","last_unstale":"2024-04-24T18:07:51.657287+0000","last_undegraded":"2024-04-24T18:07:51.657287+0000","last_fullsized":"2024-04-24T18:07:51.657287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:32:03.728352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":5652540,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1163,"num_read_kb":1761,"num_write":66,"num_write_kb":1346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"41'398","reported_seq":432,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.915428+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:07:51.915428+0000","last_peered":"2024-04-24T18:07:51.915428+0000","last_clean":"2024-04-24T18:07:51.915428+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:07:51.915428+0000","last_undegraded":"2024-04-24T18:07:51.915428+0000","last_fullsized":"2024-04-24T18:07:51.915428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":1513663,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2044,"num_read_kb":1729,"num_write":725,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"41'24","reported_seq":63,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.859672+0000","last_change":"2024-04-24T18:07:38.294088+0000","last_active":"2024-04-24T18:07:51.859672+0000","last_peered":"2024-04-24T18:07:51.859672+0000","last_clean":"2024-04-24T18:07:51.859672+0000","last_became_active":"2024-04-24T18:07:37.294537+0000","last_became_peered":"2024-04-24T18:07:37.294537+0000","last_unstale":"2024-04-24T18:07:51.859672+0000","last_undegraded":"2024-04-24T18:07:51.859672+0000","last_fullsized":"2024-04-24T18:07:51.859672+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:27:37.724978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":810728,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":24,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"41'40","reported_seq":1989,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.706582+0000","last_change":"2024-04-24T18:07:38.296551+0000","last_active":"2024-04-24T18:07:51.706582+0000","last_peered":"2024-04-24T18:07:51.706582+0000","last_clean":"2024-04-24T18:07:51.706582+0000","last_became_active":"2024-04-24T18:07:37.288733+0000","last_became_peered":"2024-04-24T18:07:37.288733+0000","last_unstale":"2024-04-24T18:07:51.706582+0000","last_undegraded":"2024-04-24T18:07:51.706582+0000","last_fullsized":"2024-04-24T18:07:51.706582+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:50.832005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":12516002,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":1946,"num_read_kb":3227,"num_write":39,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"41'44","reported_seq":2239,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.840103+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:07:51.840103+0000","last_peered":"2024-04-24T18:07:51.840103+0000","last_clean":"2024-04-24T18:07:51.840103+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:07:51.840103+0000","last_undegraded":"2024-04-24T18:07:51.840103+0000","last_fullsized":"2024-04-24T18:07:51.840103+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":9385508,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2182,"num_read_kb":1527,"num_write":49,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"41'58","reported_seq":2690,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.896417+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:07:51.896417+0000","last_peered":"2024-04-24T18:07:51.896417+0000","last_clean":"2024-04-24T18:07:51.896417+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:07:51.896417+0000","last_undegraded":"2024-04-24T18:07:51.896417+0000","last_fullsized":"2024-04-24T18:07:51.896417+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":8548672,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":2625,"num_read_kb":4092,"num_write":60,"num_write_kb":2644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"41'51","reported_seq":483,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.809482+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:07:51.809482+0000","last_peered":"2024-04-24T18:07:51.809482+0000","last_clean":"2024-04-24T18:07:51.809482+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:07:51.809482+0000","last_undegraded":"2024-04-24T18:07:51.809482+0000","last_fullsized":"2024-04-24T18:07:51.809482+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":9446241,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":412,"num_read_kb":1349,"num_write":54,"num_write_kb":855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"41'42","reported_seq":1991,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.905365+0000","last_change":"2024-04-24T18:07:38.291282+0000","last_active":"2024-04-24T18:07:51.905365+0000","last_peered":"2024-04-24T18:07:51.905365+0000","last_clean":"2024-04-24T18:07:51.905365+0000","last_became_active":"2024-04-24T18:07:36.548455+0000","last_became_peered":"2024-04-24T18:07:36.548455+0000","last_unstale":"2024-04-24T18:07:51.905365+0000","last_undegraded":"2024-04-24T18:07:51.905365+0000","last_fullsized":"2024-04-24T18:07:51.905365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:50.006764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":7997827,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1936,"num_read_kb":4882,"num_write":46,"num_write_kb":2595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"41'38","reported_seq":77,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.867231+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:07:51.867231+0000","last_peered":"2024-04-24T18:07:51.867231+0000","last_clean":"2024-04-24T18:07:51.867231+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:07:51.867231+0000","last_undegraded":"2024-04-24T18:07:51.867231+0000","last_fullsized":"2024-04-24T18:07:51.867231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":2213247,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":80,"num_write":42,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"41'36","reported_seq":163,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.835686+0000","last_change":"2024-04-24T18:07:38.299855+0000","last_active":"2024-04-24T18:07:51.835686+0000","last_peered":"2024-04-24T18:07:51.835686+0000","last_clean":"2024-04-24T18:07:51.835686+0000","last_became_active":"2024-04-24T18:07:37.294590+0000","last_became_peered":"2024-04-24T18:07:37.294590+0000","last_unstale":"2024-04-24T18:07:51.835686+0000","last_undegraded":"2024-04-24T18:07:51.835686+0000","last_fullsized":"2024-04-24T18:07:51.835686+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:00.428425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":10671128,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":118,"num_read_kb":208,"num_write":40,"num_write_kb":765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.587231+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:07:50.587231+0000","last_peered":"2024-04-24T18:07:50.587231+0000","last_clean":"2024-04-24T18:07:50.587231+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:07:50.587231+0000","last_undegraded":"2024-04-24T18:07:50.587231+0000","last_fullsized":"2024-04-24T18:07:50.587231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.587086+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:07:50.587086+0000","last_peered":"2024-04-24T18:07:50.587086+0000","last_clean":"2024-04-24T18:07:50.587086+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:07:50.587086+0000","last_undegraded":"2024-04-24T18:07:50.587086+0000","last_fullsized":"2024-04-24T18:07:50.587086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"41'52","reported_seq":1772,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.844047+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:07:51.844047+0000","last_peered":"2024-04-24T18:07:51.844047+0000","last_clean":"2024-04-24T18:07:51.844047+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:07:51.844047+0000","last_undegraded":"2024-04-24T18:07:51.844047+0000","last_fullsized":"2024-04-24T18:07:51.844047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":18393535,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1712,"num_read_kb":3480,"num_write":59,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.594319+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:07:50.594319+0000","last_peered":"2024-04-24T18:07:50.594319+0000","last_clean":"2024-04-24T18:07:50.594319+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:07:50.594319+0000","last_undegraded":"2024-04-24T18:07:50.594319+0000","last_fullsized":"2024-04-24T18:07:50.594319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.674538+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:07:50.674538+0000","last_peered":"2024-04-24T18:07:50.674538+0000","last_clean":"2024-04-24T18:07:50.674538+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:07:50.674538+0000","last_undegraded":"2024-04-24T18:07:50.674538+0000","last_fullsized":"2024-04-24T18:07:50.674538+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"41'70","reported_seq":100,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.867241+0000","last_change":"2024-04-24T18:07:38.295890+0000","last_active":"2024-04-24T18:07:51.867241+0000","last_peered":"2024-04-24T18:07:51.867241+0000","last_clean":"2024-04-24T18:07:51.867241+0000","last_became_active":"2024-04-24T18:07:37.288825+0000","last_became_peered":"2024-04-24T18:07:37.288825+0000","last_unstale":"2024-04-24T18:07:51.867241+0000","last_undegraded":"2024-04-24T18:07:51.867241+0000","last_fullsized":"2024-04-24T18:07:51.867241+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:17:02.480539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":1985328,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":129,"num_read_kb":96,"num_write":85,"num_write_kb":507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.643428+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:07:50.643428+0000","last_peered":"2024-04-24T18:07:50.643428+0000","last_clean":"2024-04-24T18:07:50.643428+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:07:50.643428+0000","last_undegraded":"2024-04-24T18:07:50.643428+0000","last_fullsized":"2024-04-24T18:07:50.643428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:13:26.133189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.640960+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:07:50.640960+0000","last_peered":"2024-04-24T18:07:50.640960+0000","last_clean":"2024-04-24T18:07:50.640960+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:07:50.640960+0000","last_undegraded":"2024-04-24T18:07:50.640960+0000","last_fullsized":"2024-04-24T18:07:50.640960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"41'345","reported_seq":3172,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.822327+0000","last_change":"2024-04-24T18:07:38.288298+0000","last_active":"2024-04-24T18:07:51.822327+0000","last_peered":"2024-04-24T18:07:51.822327+0000","last_clean":"2024-04-24T18:07:51.822327+0000","last_became_active":"2024-04-24T18:07:36.549561+0000","last_became_peered":"2024-04-24T18:07:36.549561+0000","last_unstale":"2024-04-24T18:07:51.822327+0000","last_undegraded":"2024-04-24T18:07:51.822327+0000","last_fullsized":"2024-04-24T18:07:51.822327+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:48.800593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":10710879,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":5108,"num_read_kb":9600,"num_write":498,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.699323+0000","last_change":"2024-04-24T18:07:37.291871+0000","last_active":"2024-04-24T18:07:50.699323+0000","last_peered":"2024-04-24T18:07:50.699323+0000","last_clean":"2024-04-24T18:07:50.699323+0000","last_became_active":"2024-04-24T18:07:37.291409+0000","last_became_peered":"2024-04-24T18:07:37.291409+0000","last_unstale":"2024-04-24T18:07:50.699323+0000","last_undegraded":"2024-04-24T18:07:50.699323+0000","last_fullsized":"2024-04-24T18:07:50.699323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:26.601769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.641095+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:07:50.641095+0000","last_peered":"2024-04-24T18:07:50.641095+0000","last_clean":"2024-04-24T18:07:50.641095+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:07:50.641095+0000","last_undegraded":"2024-04-24T18:07:50.641095+0000","last_fullsized":"2024-04-24T18:07:50.641095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"41'52","reported_seq":899,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.889999+0000","last_change":"2024-04-24T18:07:38.299673+0000","last_active":"2024-04-24T18:07:51.889999+0000","last_peered":"2024-04-24T18:07:51.889999+0000","last_clean":"2024-04-24T18:07:51.889999+0000","last_became_active":"2024-04-24T18:07:37.289670+0000","last_became_peered":"2024-04-24T18:07:37.289670+0000","last_unstale":"2024-04-24T18:07:51.889999+0000","last_undegraded":"2024-04-24T18:07:51.889999+0000","last_fullsized":"2024-04-24T18:07:51.889999+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:44.796031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":3051981,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1091,"num_read_kb":934,"num_write":50,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.588841+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:07:50.588841+0000","last_peered":"2024-04-24T18:07:50.588841+0000","last_clean":"2024-04-24T18:07:50.588841+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:07:50.588841+0000","last_undegraded":"2024-04-24T18:07:50.588841+0000","last_fullsized":"2024-04-24T18:07:50.588841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:25.729450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":206,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.641050+0000","last_change":"2024-04-24T18:07:40.295406+0000","last_active":"2024-04-24T18:07:50.641050+0000","last_peered":"2024-04-24T18:07:50.641050+0000","last_clean":"2024-04-24T18:07:50.641050+0000","last_became_active":"2024-04-24T18:07:40.295064+0000","last_became_peered":"2024-04-24T18:07:40.295064+0000","last_unstale":"2024-04-24T18:07:50.641050+0000","last_undegraded":"2024-04-24T18:07:50.641050+0000","last_fullsized":"2024-04-24T18:07:50.641050+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:10.525802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":25,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.587307+0000","last_change":"2024-04-24T18:07:37.295474+0000","last_active":"2024-04-24T18:07:50.587307+0000","last_peered":"2024-04-24T18:07:50.587307+0000","last_clean":"2024-04-24T18:07:50.587307+0000","last_became_active":"2024-04-24T18:07:37.295175+0000","last_became_peered":"2024-04-24T18:07:37.295175+0000","last_unstale":"2024-04-24T18:07:50.587307+0000","last_undegraded":"2024-04-24T18:07:50.587307+0000","last_fullsized":"2024-04-24T18:07:50.587307+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:50:51.540025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"41'51","reported_seq":3685,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.846360+0000","last_change":"2024-04-24T18:07:38.289378+0000","last_active":"2024-04-24T18:07:51.846360+0000","last_peered":"2024-04-24T18:07:51.846360+0000","last_clean":"2024-04-24T18:07:51.846360+0000","last_became_active":"2024-04-24T18:07:37.288708+0000","last_became_peered":"2024-04-24T18:07:37.288708+0000","last_unstale":"2024-04-24T18:07:51.846360+0000","last_undegraded":"2024-04-24T18:07:51.846360+0000","last_fullsized":"2024-04-24T18:07:51.846360+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:37.490208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":18448384,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3623,"num_read_kb":5618,"num_write":58,"num_write_kb":3182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"41'34","reported_seq":124,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.654779+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:07:51.654779+0000","last_peered":"2024-04-24T18:07:51.654779+0000","last_clean":"2024-04-24T18:07:51.654779+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:07:51.654779+0000","last_undegraded":"2024-04-24T18:07:51.654779+0000","last_fullsized":"2024-04-24T18:07:51.654779+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":9061705,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":119,"num_read_kb":66,"num_write":40,"num_write_kb":673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.643648+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:07:50.643648+0000","last_peered":"2024-04-24T18:07:50.643648+0000","last_clean":"2024-04-24T18:07:50.643648+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:07:50.643648+0000","last_undegraded":"2024-04-24T18:07:50.643648+0000","last_fullsized":"2024-04-24T18:07:50.643648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"41'54","reported_seq":476,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.867230+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:07:51.867230+0000","last_peered":"2024-04-24T18:07:51.867230+0000","last_clean":"2024-04-24T18:07:51.867230+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:07:51.867230+0000","last_undegraded":"2024-04-24T18:07:51.867230+0000","last_fullsized":"2024-04-24T18:07:51.867230+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":16653053,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":422,"num_read_kb":2294,"num_write":55,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.594315+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:07:50.594315+0000","last_peered":"2024-04-24T18:07:50.594315+0000","last_clean":"2024-04-24T18:07:50.594315+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:07:50.594315+0000","last_undegraded":"2024-04-24T18:07:50.594315+0000","last_fullsized":"2024-04-24T18:07:50.594315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"41'51","reported_seq":1898,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.904636+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:07:51.904636+0000","last_peered":"2024-04-24T18:07:51.904636+0000","last_clean":"2024-04-24T18:07:51.904636+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:07:51.904636+0000","last_undegraded":"2024-04-24T18:07:51.904636+0000","last_fullsized":"2024-04-24T18:07:51.904636+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":12386905,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1842,"num_read_kb":2926,"num_write":54,"num_write_kb":1396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"41'60","reported_seq":1264,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.743449+0000","last_change":"2024-04-24T18:07:38.291271+0000","last_active":"2024-04-24T18:07:51.743449+0000","last_peered":"2024-04-24T18:07:51.743449+0000","last_clean":"2024-04-24T18:07:51.743449+0000","last_became_active":"2024-04-24T18:07:37.294321+0000","last_became_peered":"2024-04-24T18:07:37.294321+0000","last_unstale":"2024-04-24T18:07:51.743449+0000","last_undegraded":"2024-04-24T18:07:51.743449+0000","last_fullsized":"2024-04-24T18:07:51.743449+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:03.283317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":12313001,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1381,"num_read_kb":2446,"num_write":93,"num_write_kb":1299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"41'49","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.860356+0000","last_change":"2024-04-24T18:07:38.288318+0000","last_active":"2024-04-24T18:07:51.860356+0000","last_peered":"2024-04-24T18:07:51.860356+0000","last_clean":"2024-04-24T18:07:51.860356+0000","last_became_active":"2024-04-24T18:07:37.292006+0000","last_became_peered":"2024-04-24T18:07:37.292006+0000","last_unstale":"2024-04-24T18:07:51.860356+0000","last_undegraded":"2024-04-24T18:07:51.860356+0000","last_fullsized":"2024-04-24T18:07:51.860356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:49:37.784848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":10175762,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":79,"num_write":56,"num_write_kb":1902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"41'33","reported_seq":120,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.747270+0000","last_change":"2024-04-24T18:07:38.289002+0000","last_active":"2024-04-24T18:07:51.747270+0000","last_peered":"2024-04-24T18:07:51.747270+0000","last_clean":"2024-04-24T18:07:51.747270+0000","last_became_active":"2024-04-24T18:07:37.294805+0000","last_became_peered":"2024-04-24T18:07:37.294805+0000","last_unstale":"2024-04-24T18:07:51.747270+0000","last_undegraded":"2024-04-24T18:07:51.747270+0000","last_fullsized":"2024-04-24T18:07:51.747270+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:19.894060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":5407798,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":81,"num_write":34,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"41'46","reported_seq":264,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.846270+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:07:51.846270+0000","last_peered":"2024-04-24T18:07:51.846270+0000","last_clean":"2024-04-24T18:07:51.846270+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:07:51.846270+0000","last_undegraded":"2024-04-24T18:07:51.846270+0000","last_fullsized":"2024-04-24T18:07:51.846270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":9386802,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":209,"num_read_kb":75,"num_write":50,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"41'29","reported_seq":58,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.915080+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:07:51.915080+0000","last_peered":"2024-04-24T18:07:51.915080+0000","last_clean":"2024-04-24T18:07:51.915080+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:07:51.915080+0000","last_undegraded":"2024-04-24T18:07:51.915080+0000","last_fullsized":"2024-04-24T18:07:51.915080+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":5226806,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":54,"num_write":29,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"41'78","reported_seq":302,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.923085+0000","last_change":"2024-04-24T18:07:38.291549+0000","last_active":"2024-04-24T18:07:51.923085+0000","last_peered":"2024-04-24T18:07:51.923085+0000","last_clean":"2024-04-24T18:07:51.923085+0000","last_became_active":"2024-04-24T18:07:36.549997+0000","last_became_peered":"2024-04-24T18:07:36.549997+0000","last_unstale":"2024-04-24T18:07:51.923085+0000","last_undegraded":"2024-04-24T18:07:51.923085+0000","last_fullsized":"2024-04-24T18:07:51.923085+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:15.050671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":1476608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":527,"num_read_kb":535,"num_write":71,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"41'52","reported_seq":354,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.745028+0000","last_change":"2024-04-24T18:07:38.294105+0000","last_active":"2024-04-24T18:07:51.745028+0000","last_peered":"2024-04-24T18:07:51.745028+0000","last_clean":"2024-04-24T18:07:51.745028+0000","last_became_active":"2024-04-24T18:07:37.294672+0000","last_became_peered":"2024-04-24T18:07:37.294672+0000","last_unstale":"2024-04-24T18:07:51.745028+0000","last_undegraded":"2024-04-24T18:07:51.745028+0000","last_fullsized":"2024-04-24T18:07:51.745028+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:00:08.672989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":2486272,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":282,"num_read_kb":2231,"num_write":51,"num_write_kb":702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"41'52","reported_seq":90,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.816897+0000","last_change":"2024-04-24T18:07:38.293837+0000","last_active":"2024-04-24T18:07:51.816897+0000","last_peered":"2024-04-24T18:07:51.816897+0000","last_clean":"2024-04-24T18:07:51.816897+0000","last_became_active":"2024-04-24T18:07:37.291557+0000","last_became_peered":"2024-04-24T18:07:37.291557+0000","last_unstale":"2024-04-24T18:07:51.816897+0000","last_undegraded":"2024-04-24T18:07:51.816897+0000","last_fullsized":"2024-04-24T18:07:51.816897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:10:59.686324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":2051571,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":41,"num_write":53,"num_write_kb":918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"41'103","reported_seq":763,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.761206+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:07:51.761206+0000","last_peered":"2024-04-24T18:07:51.761206+0000","last_clean":"2024-04-24T18:07:51.761206+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:07:51.761206+0000","last_undegraded":"2024-04-24T18:07:51.761206+0000","last_fullsized":"2024-04-24T18:07:51.761206+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":6668604,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1250,"num_read_kb":1609,"num_write":102,"num_write_kb":962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"41'57","reported_seq":1467,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.896686+0000","last_change":"2024-04-24T18:07:38.288507+0000","last_active":"2024-04-24T18:07:51.896686+0000","last_peered":"2024-04-24T18:07:51.896686+0000","last_clean":"2024-04-24T18:07:51.896686+0000","last_became_active":"2024-04-24T18:07:37.291286+0000","last_became_peered":"2024-04-24T18:07:37.291286+0000","last_unstale":"2024-04-24T18:07:51.896686+0000","last_undegraded":"2024-04-24T18:07:51.896686+0000","last_fullsized":"2024-04-24T18:07:51.896686+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:33.189077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":5985741,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1617,"num_read_kb":2479,"num_write":52,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"41'44","reported_seq":2144,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.808240+0000","last_change":"2024-04-24T18:07:38.292302+0000","last_active":"2024-04-24T18:07:51.808240+0000","last_peered":"2024-04-24T18:07:51.808240+0000","last_clean":"2024-04-24T18:07:51.808240+0000","last_became_active":"2024-04-24T18:07:36.549147+0000","last_became_peered":"2024-04-24T18:07:36.549147+0000","last_unstale":"2024-04-24T18:07:51.808240+0000","last_undegraded":"2024-04-24T18:07:51.808240+0000","last_fullsized":"2024-04-24T18:07:51.808240+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:02.455780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":2486852,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2089,"num_read_kb":1415,"num_write":46,"num_write_kb":2082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"41'48","reported_seq":249,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.752971+0000","last_change":"2024-04-24T18:07:38.294357+0000","last_active":"2024-04-24T18:07:51.752971+0000","last_peered":"2024-04-24T18:07:51.752971+0000","last_clean":"2024-04-24T18:07:51.752971+0000","last_became_active":"2024-04-24T18:07:37.290498+0000","last_became_peered":"2024-04-24T18:07:37.290498+0000","last_unstale":"2024-04-24T18:07:51.752971+0000","last_undegraded":"2024-04-24T18:07:51.752971+0000","last_fullsized":"2024-04-24T18:07:51.752971+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:26.264283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":3312227,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":429,"num_write":48,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"41'44","reported_seq":732,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.869524+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:07:51.869524+0000","last_peered":"2024-04-24T18:07:51.869524+0000","last_clean":"2024-04-24T18:07:51.869524+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:07:51.869524+0000","last_undegraded":"2024-04-24T18:07:51.869524+0000","last_fullsized":"2024-04-24T18:07:51.869524+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":4206240,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":672,"num_read_kb":1040,"num_write":50,"num_write_kb":2136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"41'390","reported_seq":484,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.661247+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:07:51.661247+0000","last_peered":"2024-04-24T18:07:51.661247+0000","last_clean":"2024-04-24T18:07:51.661247+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:07:51.661247+0000","last_undegraded":"2024-04-24T18:07:51.661247+0000","last_fullsized":"2024-04-24T18:07:51.661247+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":6584406,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1882,"num_read_kb":1537,"num_write":638,"num_write_kb":830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":237216016,"num_objects":218,"num_object_clones":12,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":218,"num_whiteouts":9,"num_read":37046,"num_read_kb":57920,"num_write":3417,"num_write_kb":39599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8962048,"data_stored":66869926,"data_compressed":1825202,"data_compressed_allocated":4841472,"data_compressed_original":63826196,"omap_allocated":0,"internal_metadata":0},"log_size":2589,"ondisk_log_size":2589,"up":64,"acting":64,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":27030,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1572864,"data_stored":7745270,"data_compressed":233576,"data_compressed_allocated":684032,"data_compressed_original":7273956,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303040,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1622016,"data_stored":10231388,"data_compressed":237941,"data_compressed_allocated":667648,"data_compressed_original":9654272,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]}]},{"osd":1,"up_from":0,"seq":55834574857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302996,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068844,"statfs":{"total":96636764160,"available":96326496256,"internally_reserved":0,"allocated":1585152,"data_stored":9580651,"data_compressed":264285,"data_compressed_allocated":643072,"data_compressed_original":9007736,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1208320,"data_stored":5576817,"data_compressed":199145,"data_compressed_allocated":356352,"data_compressed_original":5046272,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.219}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303544,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068296,"statfs":{"total":96636764160,"available":96325935104,"internally_reserved":0,"allocated":2117632,"data_stored":11906306,"data_compressed":326848,"data_compressed_allocated":876544,"data_compressed_original":11159772,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2453504,"data_stored":16493406,"data_compressed":412059,"data_compressed_allocated":1064960,"data_compressed_original":15581184,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298948,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072892,"statfs":{"total":96636764160,"available":96330641408,"internally_reserved":0,"allocated":1622016,"data_stored":7888449,"data_compressed":169339,"data_compressed_allocated":602112,"data_compressed_original":7263690,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081344,"data_stored":8785700,"data_compressed":254583,"data_compressed_allocated":606208,"data_compressed_original":8417912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":745472,"data_stored":5372234,"data_compressed":199145,"data_compressed_allocated":356352,"data_compressed_original":5046272,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1654784,"data_stored":11701723,"data_compressed":326848,"data_compressed_allocated":876544,"data_compressed_original":11159772,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1990656,"data_stored":16288823,"data_compressed":412059,"data_compressed_allocated":1064960,"data_compressed_original":15581184,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1179648,"data_stored":7698600,"data_compressed":169418,"data_compressed_allocated":606208,"data_compressed_original":7271882,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1257472,"data_stored":10072527,"data_compressed":238237,"data_compressed_allocated":684032,"data_compressed_original":9648658,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1052672,"data_stored":6950319,"data_compressed":224912,"data_compressed_allocated":647168,"data_compressed_original":6700516,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:07:54.085 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:07:54.124 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:07:54.163 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:54.171+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:54.199 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:54.199 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:54.200 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:54.210 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:07:54.273 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:54.273 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:54.281+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:54.293 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:54.312 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:54.331 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:54.350 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:54.364 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:54.372+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:54.369 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:54.383 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:54.384 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:54.388 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:54.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:07:54.407 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:54.409 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:54.409 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:54.409 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:54.410 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:54.410 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:54.410 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:54.410 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:54.410 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:54.465 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:54.473+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:54.469 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:07:54.471 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:07:54.482 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":54,"stamp":"2024-04-24T18:07:53.606487+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":237806384,"num_objects":220,"num_object_clones":12,"num_object_copies":440,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":220,"num_whiteouts":9,"num_read":37162,"num_read_kb":58020,"num_write":3576,"num_write_kb":42015,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2706,"ondisk_log_size":2706,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":7,"num_per_pool_osds":7,"num_per_pool_omap_osds":7,"kb":660602880,"kb_used":2113908,"kb_used_data":11896,"kb_used_omap":11,"kb_used_meta":2101876,"kb_avail":658488972,"statfs":{"total":676457349120,"available":674292707328,"internally_reserved":0,"allocated":12181504,"data_stored":69422287,"data_compressed":1843193,"data_compressed_allocated":4894720,"data_compressed_original":64986882,"omap_allocated":11928,"internal_metadata":2152321384},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":68018896,"num_objects":121,"num_object_clones":12,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":9,"num_read":28973,"num_read_kb":50112,"num_write":1622,"num_write_kb":31389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.021738"},"pg_stats":[{"pgid":"4.1f","version":"41'64","reported_seq":935,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.657287+0000","last_change":"2024-04-24T18:07:38.288543+0000","last_active":"2024-04-24T18:07:51.657287+0000","last_peered":"2024-04-24T18:07:51.657287+0000","last_clean":"2024-04-24T18:07:51.657287+0000","last_became_active":"2024-04-24T18:07:36.550017+0000","last_became_peered":"2024-04-24T18:07:36.550017+0000","last_unstale":"2024-04-24T18:07:51.657287+0000","last_undegraded":"2024-04-24T18:07:51.657287+0000","last_fullsized":"2024-04-24T18:07:51.657287+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":64,"log_dups_size":0,"ondisk_log_size":64,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:32:03.728352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":5652540,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1163,"num_read_kb":1761,"num_write":66,"num_write_kb":1346,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"41'398","reported_seq":432,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.915428+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:07:51.915428+0000","last_peered":"2024-04-24T18:07:51.915428+0000","last_clean":"2024-04-24T18:07:51.915428+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:07:51.915428+0000","last_undegraded":"2024-04-24T18:07:51.915428+0000","last_fullsized":"2024-04-24T18:07:51.915428+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":398,"log_dups_size":0,"ondisk_log_size":398,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":1513663,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2044,"num_read_kb":1729,"num_write":725,"num_write_kb":2176,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"41'24","reported_seq":63,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.859672+0000","last_change":"2024-04-24T18:07:38.294088+0000","last_active":"2024-04-24T18:07:51.859672+0000","last_peered":"2024-04-24T18:07:51.859672+0000","last_clean":"2024-04-24T18:07:51.859672+0000","last_became_active":"2024-04-24T18:07:37.294537+0000","last_became_peered":"2024-04-24T18:07:37.294537+0000","last_unstale":"2024-04-24T18:07:51.859672+0000","last_undegraded":"2024-04-24T18:07:51.859672+0000","last_fullsized":"2024-04-24T18:07:51.859672+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:27:37.724978+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":810728,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":0,"num_write":24,"num_write_kb":99,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"41'40","reported_seq":1989,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.706582+0000","last_change":"2024-04-24T18:07:38.296551+0000","last_active":"2024-04-24T18:07:51.706582+0000","last_peered":"2024-04-24T18:07:51.706582+0000","last_clean":"2024-04-24T18:07:51.706582+0000","last_became_active":"2024-04-24T18:07:37.288733+0000","last_became_peered":"2024-04-24T18:07:37.288733+0000","last_unstale":"2024-04-24T18:07:51.706582+0000","last_undegraded":"2024-04-24T18:07:51.706582+0000","last_fullsized":"2024-04-24T18:07:51.706582+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:26:50.832005+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":12516002,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":1946,"num_read_kb":3227,"num_write":39,"num_write_kb":199,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"41'44","reported_seq":2239,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.840103+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:07:51.840103+0000","last_peered":"2024-04-24T18:07:51.840103+0000","last_clean":"2024-04-24T18:07:51.840103+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:07:51.840103+0000","last_undegraded":"2024-04-24T18:07:51.840103+0000","last_fullsized":"2024-04-24T18:07:51.840103+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":9385508,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":2182,"num_read_kb":1527,"num_write":49,"num_write_kb":2765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"41'58","reported_seq":2690,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.896417+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:07:51.896417+0000","last_peered":"2024-04-24T18:07:51.896417+0000","last_clean":"2024-04-24T18:07:51.896417+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:07:51.896417+0000","last_undegraded":"2024-04-24T18:07:51.896417+0000","last_fullsized":"2024-04-24T18:07:51.896417+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":8548672,"num_objects":9,"num_object_clones":2,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":2,"num_read":2625,"num_read_kb":4092,"num_write":60,"num_write_kb":2644,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"41'51","reported_seq":483,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.809482+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:07:51.809482+0000","last_peered":"2024-04-24T18:07:51.809482+0000","last_clean":"2024-04-24T18:07:51.809482+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:07:51.809482+0000","last_undegraded":"2024-04-24T18:07:51.809482+0000","last_fullsized":"2024-04-24T18:07:51.809482+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":9446241,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":412,"num_read_kb":1349,"num_write":54,"num_write_kb":855,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"41'42","reported_seq":1991,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.905365+0000","last_change":"2024-04-24T18:07:38.291282+0000","last_active":"2024-04-24T18:07:51.905365+0000","last_peered":"2024-04-24T18:07:51.905365+0000","last_clean":"2024-04-24T18:07:51.905365+0000","last_became_active":"2024-04-24T18:07:36.548455+0000","last_became_peered":"2024-04-24T18:07:36.548455+0000","last_unstale":"2024-04-24T18:07:51.905365+0000","last_undegraded":"2024-04-24T18:07:51.905365+0000","last_fullsized":"2024-04-24T18:07:51.905365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:50.006764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":7997827,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1936,"num_read_kb":4882,"num_write":46,"num_write_kb":2595,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"41'38","reported_seq":77,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.867231+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:07:51.867231+0000","last_peered":"2024-04-24T18:07:51.867231+0000","last_clean":"2024-04-24T18:07:51.867231+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:07:51.867231+0000","last_undegraded":"2024-04-24T18:07:51.867231+0000","last_fullsized":"2024-04-24T18:07:51.867231+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":2213247,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":28,"num_read_kb":80,"num_write":42,"num_write_kb":798,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"41'36","reported_seq":163,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.835686+0000","last_change":"2024-04-24T18:07:38.299855+0000","last_active":"2024-04-24T18:07:51.835686+0000","last_peered":"2024-04-24T18:07:51.835686+0000","last_clean":"2024-04-24T18:07:51.835686+0000","last_became_active":"2024-04-24T18:07:37.294590+0000","last_became_peered":"2024-04-24T18:07:37.294590+0000","last_unstale":"2024-04-24T18:07:51.835686+0000","last_undegraded":"2024-04-24T18:07:51.835686+0000","last_fullsized":"2024-04-24T18:07:51.835686+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:00.428425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":10671128,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":118,"num_read_kb":208,"num_write":40,"num_write_kb":765,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.587231+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:07:50.587231+0000","last_peered":"2024-04-24T18:07:50.587231+0000","last_clean":"2024-04-24T18:07:50.587231+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:07:50.587231+0000","last_undegraded":"2024-04-24T18:07:50.587231+0000","last_fullsized":"2024-04-24T18:07:50.587231+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.587086+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:07:50.587086+0000","last_peered":"2024-04-24T18:07:50.587086+0000","last_clean":"2024-04-24T18:07:50.587086+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:07:50.587086+0000","last_undegraded":"2024-04-24T18:07:50.587086+0000","last_fullsized":"2024-04-24T18:07:50.587086+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"41'52","reported_seq":1772,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.844047+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:07:51.844047+0000","last_peered":"2024-04-24T18:07:51.844047+0000","last_clean":"2024-04-24T18:07:51.844047+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:07:51.844047+0000","last_undegraded":"2024-04-24T18:07:51.844047+0000","last_fullsized":"2024-04-24T18:07:51.844047+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":18393535,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":1712,"num_read_kb":3480,"num_write":59,"num_write_kb":3357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.594319+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:07:50.594319+0000","last_peered":"2024-04-24T18:07:50.594319+0000","last_clean":"2024-04-24T18:07:50.594319+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:07:50.594319+0000","last_undegraded":"2024-04-24T18:07:50.594319+0000","last_fullsized":"2024-04-24T18:07:50.594319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.674538+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:07:50.674538+0000","last_peered":"2024-04-24T18:07:50.674538+0000","last_clean":"2024-04-24T18:07:50.674538+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:07:50.674538+0000","last_undegraded":"2024-04-24T18:07:50.674538+0000","last_fullsized":"2024-04-24T18:07:50.674538+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"41'70","reported_seq":100,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.867241+0000","last_change":"2024-04-24T18:07:38.295890+0000","last_active":"2024-04-24T18:07:51.867241+0000","last_peered":"2024-04-24T18:07:51.867241+0000","last_clean":"2024-04-24T18:07:51.867241+0000","last_became_active":"2024-04-24T18:07:37.288825+0000","last_became_peered":"2024-04-24T18:07:37.288825+0000","last_unstale":"2024-04-24T18:07:51.867241+0000","last_undegraded":"2024-04-24T18:07:51.867241+0000","last_fullsized":"2024-04-24T18:07:51.867241+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:17:02.480539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":1985328,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":1,"num_read":129,"num_read_kb":96,"num_write":85,"num_write_kb":507,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.643428+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:07:50.643428+0000","last_peered":"2024-04-24T18:07:50.643428+0000","last_clean":"2024-04-24T18:07:50.643428+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:07:50.643428+0000","last_undegraded":"2024-04-24T18:07:50.643428+0000","last_fullsized":"2024-04-24T18:07:50.643428+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:13:26.133189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.640960+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:07:50.640960+0000","last_peered":"2024-04-24T18:07:50.640960+0000","last_clean":"2024-04-24T18:07:50.640960+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:07:50.640960+0000","last_undegraded":"2024-04-24T18:07:50.640960+0000","last_fullsized":"2024-04-24T18:07:50.640960+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"41'345","reported_seq":3172,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.822327+0000","last_change":"2024-04-24T18:07:38.288298+0000","last_active":"2024-04-24T18:07:51.822327+0000","last_peered":"2024-04-24T18:07:51.822327+0000","last_clean":"2024-04-24T18:07:51.822327+0000","last_became_active":"2024-04-24T18:07:36.549561+0000","last_became_peered":"2024-04-24T18:07:36.549561+0000","last_unstale":"2024-04-24T18:07:51.822327+0000","last_undegraded":"2024-04-24T18:07:51.822327+0000","last_fullsized":"2024-04-24T18:07:51.822327+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":345,"log_dups_size":0,"ondisk_log_size":345,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:48.800593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":10710879,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":5108,"num_read_kb":9600,"num_write":498,"num_write_kb":1011,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":25,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.699323+0000","last_change":"2024-04-24T18:07:37.291871+0000","last_active":"2024-04-24T18:07:50.699323+0000","last_peered":"2024-04-24T18:07:50.699323+0000","last_clean":"2024-04-24T18:07:50.699323+0000","last_became_active":"2024-04-24T18:07:37.291409+0000","last_became_peered":"2024-04-24T18:07:37.291409+0000","last_unstale":"2024-04-24T18:07:50.699323+0000","last_undegraded":"2024-04-24T18:07:50.699323+0000","last_fullsized":"2024-04-24T18:07:50.699323+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:12:26.601769+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":32,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.641095+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:07:50.641095+0000","last_peered":"2024-04-24T18:07:50.641095+0000","last_clean":"2024-04-24T18:07:50.641095+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:07:50.641095+0000","last_undegraded":"2024-04-24T18:07:50.641095+0000","last_fullsized":"2024-04-24T18:07:50.641095+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"41'52","reported_seq":899,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.889999+0000","last_change":"2024-04-24T18:07:38.299673+0000","last_active":"2024-04-24T18:07:51.889999+0000","last_peered":"2024-04-24T18:07:51.889999+0000","last_clean":"2024-04-24T18:07:51.889999+0000","last_became_active":"2024-04-24T18:07:37.289670+0000","last_became_peered":"2024-04-24T18:07:37.289670+0000","last_unstale":"2024-04-24T18:07:51.889999+0000","last_undegraded":"2024-04-24T18:07:51.889999+0000","last_fullsized":"2024-04-24T18:07:51.889999+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:42:44.796031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":3051981,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1091,"num_read_kb":934,"num_write":50,"num_write_kb":270,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":30,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.588841+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:07:50.588841+0000","last_peered":"2024-04-24T18:07:50.588841+0000","last_clean":"2024-04-24T18:07:50.588841+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:07:50.588841+0000","last_undegraded":"2024-04-24T18:07:50.588841+0000","last_fullsized":"2024-04-24T18:07:50.588841+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:25.729450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":206,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.641050+0000","last_change":"2024-04-24T18:07:40.295406+0000","last_active":"2024-04-24T18:07:50.641050+0000","last_peered":"2024-04-24T18:07:50.641050+0000","last_clean":"2024-04-24T18:07:50.641050+0000","last_became_active":"2024-04-24T18:07:40.295064+0000","last_became_peered":"2024-04-24T18:07:40.295064+0000","last_unstale":"2024-04-24T18:07:50.641050+0000","last_undegraded":"2024-04-24T18:07:50.641050+0000","last_fullsized":"2024-04-24T18:07:50.641050+0000","mapping_epoch":30,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":31,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:40:10.525802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":25,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.587307+0000","last_change":"2024-04-24T18:07:37.295474+0000","last_active":"2024-04-24T18:07:50.587307+0000","last_peered":"2024-04-24T18:07:50.587307+0000","last_clean":"2024-04-24T18:07:50.587307+0000","last_became_active":"2024-04-24T18:07:37.295175+0000","last_became_peered":"2024-04-24T18:07:37.295175+0000","last_unstale":"2024-04-24T18:07:50.587307+0000","last_undegraded":"2024-04-24T18:07:50.587307+0000","last_fullsized":"2024-04-24T18:07:50.587307+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:50:51.540025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.6","version":"41'51","reported_seq":3685,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.846360+0000","last_change":"2024-04-24T18:07:38.289378+0000","last_active":"2024-04-24T18:07:51.846360+0000","last_peered":"2024-04-24T18:07:51.846360+0000","last_clean":"2024-04-24T18:07:51.846360+0000","last_became_active":"2024-04-24T18:07:37.288708+0000","last_became_peered":"2024-04-24T18:07:37.288708+0000","last_unstale":"2024-04-24T18:07:51.846360+0000","last_undegraded":"2024-04-24T18:07:51.846360+0000","last_fullsized":"2024-04-24T18:07:51.846360+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:34:37.490208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":18448384,"num_objects":9,"num_object_clones":1,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":3623,"num_read_kb":5618,"num_write":58,"num_write_kb":3182,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"41'34","reported_seq":124,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.654779+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:07:51.654779+0000","last_peered":"2024-04-24T18:07:51.654779+0000","last_clean":"2024-04-24T18:07:51.654779+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:07:51.654779+0000","last_undegraded":"2024-04-24T18:07:51.654779+0000","last_fullsized":"2024-04-24T18:07:51.654779+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":9061705,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":119,"num_read_kb":66,"num_write":40,"num_write_kb":673,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.643648+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:07:50.643648+0000","last_peered":"2024-04-24T18:07:50.643648+0000","last_clean":"2024-04-24T18:07:50.643648+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:07:50.643648+0000","last_undegraded":"2024-04-24T18:07:50.643648+0000","last_fullsized":"2024-04-24T18:07:50.643648+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"41'54","reported_seq":476,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.867230+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:07:51.867230+0000","last_peered":"2024-04-24T18:07:51.867230+0000","last_clean":"2024-04-24T18:07:51.867230+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:07:51.867230+0000","last_undegraded":"2024-04-24T18:07:51.867230+0000","last_fullsized":"2024-04-24T18:07:51.867230+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":54,"log_dups_size":0,"ondisk_log_size":54,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":16653053,"num_objects":12,"num_object_clones":1,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":12,"num_whiteouts":1,"num_read":422,"num_read_kb":2294,"num_write":55,"num_write_kb":345,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":33,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-04-24T18:07:50.594315+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:07:50.594315+0000","last_peered":"2024-04-24T18:07:50.594315+0000","last_clean":"2024-04-24T18:07:50.594315+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:07:50.594315+0000","last_undegraded":"2024-04-24T18:07:50.594315+0000","last_fullsized":"2024-04-24T18:07:50.594315+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"41'51","reported_seq":1898,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.904636+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:07:51.904636+0000","last_peered":"2024-04-24T18:07:51.904636+0000","last_clean":"2024-04-24T18:07:51.904636+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:07:51.904636+0000","last_undegraded":"2024-04-24T18:07:51.904636+0000","last_fullsized":"2024-04-24T18:07:51.904636+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":12386905,"num_objects":8,"num_object_clones":1,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":1,"num_read":1842,"num_read_kb":2926,"num_write":54,"num_write_kb":1396,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"41'60","reported_seq":1264,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.743449+0000","last_change":"2024-04-24T18:07:38.291271+0000","last_active":"2024-04-24T18:07:51.743449+0000","last_peered":"2024-04-24T18:07:51.743449+0000","last_clean":"2024-04-24T18:07:51.743449+0000","last_became_active":"2024-04-24T18:07:37.294321+0000","last_became_peered":"2024-04-24T18:07:37.294321+0000","last_unstale":"2024-04-24T18:07:51.743449+0000","last_undegraded":"2024-04-24T18:07:51.743449+0000","last_fullsized":"2024-04-24T18:07:51.743449+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:54:03.283317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":12313001,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1381,"num_read_kb":2446,"num_write":93,"num_write_kb":1299,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"41'49","reported_seq":92,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.860356+0000","last_change":"2024-04-24T18:07:38.288318+0000","last_active":"2024-04-24T18:07:51.860356+0000","last_peered":"2024-04-24T18:07:51.860356+0000","last_clean":"2024-04-24T18:07:51.860356+0000","last_became_active":"2024-04-24T18:07:37.292006+0000","last_became_peered":"2024-04-24T18:07:37.292006+0000","last_unstale":"2024-04-24T18:07:51.860356+0000","last_undegraded":"2024-04-24T18:07:51.860356+0000","last_fullsized":"2024-04-24T18:07:51.860356+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:49:37.784848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":10175762,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":37,"num_read_kb":79,"num_write":56,"num_write_kb":1902,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"41'33","reported_seq":120,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.747270+0000","last_change":"2024-04-24T18:07:38.289002+0000","last_active":"2024-04-24T18:07:51.747270+0000","last_peered":"2024-04-24T18:07:51.747270+0000","last_clean":"2024-04-24T18:07:51.747270+0000","last_became_active":"2024-04-24T18:07:37.294805+0000","last_became_peered":"2024-04-24T18:07:37.294805+0000","last_unstale":"2024-04-24T18:07:51.747270+0000","last_undegraded":"2024-04-24T18:07:51.747270+0000","last_fullsized":"2024-04-24T18:07:51.747270+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:30:19.894060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":5407798,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":69,"num_read_kb":81,"num_write":34,"num_write_kb":284,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"41'46","reported_seq":264,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.846270+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:07:51.846270+0000","last_peered":"2024-04-24T18:07:51.846270+0000","last_clean":"2024-04-24T18:07:51.846270+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:07:51.846270+0000","last_undegraded":"2024-04-24T18:07:51.846270+0000","last_fullsized":"2024-04-24T18:07:51.846270+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":9386802,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":209,"num_read_kb":75,"num_write":50,"num_write_kb":1424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"41'29","reported_seq":58,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.915080+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:07:51.915080+0000","last_peered":"2024-04-24T18:07:51.915080+0000","last_clean":"2024-04-24T18:07:51.915080+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:07:51.915080+0000","last_undegraded":"2024-04-24T18:07:51.915080+0000","last_fullsized":"2024-04-24T18:07:51.915080+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":29,"log_dups_size":0,"ondisk_log_size":29,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":5226806,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":5,"num_read_kb":54,"num_write":29,"num_write_kb":305,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"41'78","reported_seq":302,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.923085+0000","last_change":"2024-04-24T18:07:38.291549+0000","last_active":"2024-04-24T18:07:51.923085+0000","last_peered":"2024-04-24T18:07:51.923085+0000","last_clean":"2024-04-24T18:07:51.923085+0000","last_became_active":"2024-04-24T18:07:36.549997+0000","last_became_peered":"2024-04-24T18:07:36.549997+0000","last_unstale":"2024-04-24T18:07:51.923085+0000","last_undegraded":"2024-04-24T18:07:51.923085+0000","last_fullsized":"2024-04-24T18:07:51.923085+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":78,"log_dups_size":0,"ondisk_log_size":78,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:15.050671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":1476608,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":527,"num_read_kb":535,"num_write":71,"num_write_kb":1288,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"41'52","reported_seq":354,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.745028+0000","last_change":"2024-04-24T18:07:38.294105+0000","last_active":"2024-04-24T18:07:51.745028+0000","last_peered":"2024-04-24T18:07:51.745028+0000","last_clean":"2024-04-24T18:07:51.745028+0000","last_became_active":"2024-04-24T18:07:37.294672+0000","last_became_peered":"2024-04-24T18:07:37.294672+0000","last_unstale":"2024-04-24T18:07:51.745028+0000","last_undegraded":"2024-04-24T18:07:51.745028+0000","last_fullsized":"2024-04-24T18:07:51.745028+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:00:08.672989+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":2486272,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":282,"num_read_kb":2231,"num_write":51,"num_write_kb":702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"41'52","reported_seq":90,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.816897+0000","last_change":"2024-04-24T18:07:38.293837+0000","last_active":"2024-04-24T18:07:51.816897+0000","last_peered":"2024-04-24T18:07:51.816897+0000","last_clean":"2024-04-24T18:07:51.816897+0000","last_became_active":"2024-04-24T18:07:37.291557+0000","last_became_peered":"2024-04-24T18:07:37.291557+0000","last_unstale":"2024-04-24T18:07:51.816897+0000","last_undegraded":"2024-04-24T18:07:51.816897+0000","last_fullsized":"2024-04-24T18:07:51.816897+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:10:59.686324+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":2051571,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":22,"num_read_kb":41,"num_write":53,"num_write_kb":918,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"41'103","reported_seq":763,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.761206+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:07:51.761206+0000","last_peered":"2024-04-24T18:07:51.761206+0000","last_clean":"2024-04-24T18:07:51.761206+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:07:51.761206+0000","last_undegraded":"2024-04-24T18:07:51.761206+0000","last_fullsized":"2024-04-24T18:07:51.761206+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":103,"log_dups_size":0,"ondisk_log_size":103,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":6668604,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1250,"num_read_kb":1609,"num_write":102,"num_write_kb":962,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"41'57","reported_seq":1467,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.896686+0000","last_change":"2024-04-24T18:07:38.288507+0000","last_active":"2024-04-24T18:07:51.896686+0000","last_peered":"2024-04-24T18:07:51.896686+0000","last_clean":"2024-04-24T18:07:51.896686+0000","last_became_active":"2024-04-24T18:07:37.291286+0000","last_became_peered":"2024-04-24T18:07:37.291286+0000","last_unstale":"2024-04-24T18:07:51.896686+0000","last_undegraded":"2024-04-24T18:07:51.896686+0000","last_fullsized":"2024-04-24T18:07:51.896686+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":57,"log_dups_size":0,"ondisk_log_size":57,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:51:33.189077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":5985741,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1617,"num_read_kb":2479,"num_write":52,"num_write_kb":306,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.13","version":"41'44","reported_seq":2144,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.808240+0000","last_change":"2024-04-24T18:07:38.292302+0000","last_active":"2024-04-24T18:07:51.808240+0000","last_peered":"2024-04-24T18:07:51.808240+0000","last_clean":"2024-04-24T18:07:51.808240+0000","last_became_active":"2024-04-24T18:07:36.549147+0000","last_became_peered":"2024-04-24T18:07:36.549147+0000","last_unstale":"2024-04-24T18:07:51.808240+0000","last_undegraded":"2024-04-24T18:07:51.808240+0000","last_fullsized":"2024-04-24T18:07:51.808240+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:02.455780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":2486852,"num_objects":5,"num_object_clones":1,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":2089,"num_read_kb":1415,"num_write":46,"num_write_kb":2082,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"41'48","reported_seq":249,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.752971+0000","last_change":"2024-04-24T18:07:38.294357+0000","last_active":"2024-04-24T18:07:51.752971+0000","last_peered":"2024-04-24T18:07:51.752971+0000","last_clean":"2024-04-24T18:07:51.752971+0000","last_became_active":"2024-04-24T18:07:37.290498+0000","last_became_peered":"2024-04-24T18:07:37.290498+0000","last_unstale":"2024-04-24T18:07:51.752971+0000","last_undegraded":"2024-04-24T18:07:51.752971+0000","last_fullsized":"2024-04-24T18:07:51.752971+0000","mapping_epoch":27,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":28,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:46:26.264283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":3312227,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":487,"num_read_kb":429,"num_write":48,"num_write_kb":178,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"41'44","reported_seq":732,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.869524+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:07:51.869524+0000","last_peered":"2024-04-24T18:07:51.869524+0000","last_clean":"2024-04-24T18:07:51.869524+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:07:51.869524+0000","last_undegraded":"2024-04-24T18:07:51.869524+0000","last_fullsized":"2024-04-24T18:07:51.869524+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":4206240,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":672,"num_read_kb":1040,"num_write":50,"num_write_kb":2136,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"41'390","reported_seq":484,"reported_epoch":41,"state":"active+clean","last_fresh":"2024-04-24T18:07:51.661247+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:07:51.661247+0000","last_peered":"2024-04-24T18:07:51.661247+0000","last_clean":"2024-04-24T18:07:51.661247+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:07:51.661247+0000","last_undegraded":"2024-04-24T18:07:51.661247+0000","last_fullsized":"2024-04-24T18:07:51.661247+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":390,"log_dups_size":0,"ondisk_log_size":390,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":6584406,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":1882,"num_read_kb":1537,"num_write":638,"num_write_kb":830,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":237216016,"num_objects":218,"num_object_clones":12,"num_object_copies":436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":218,"num_whiteouts":9,"num_read":37046,"num_read_kb":57920,"num_write":3417,"num_write_kb":39599,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8962048,"data_stored":66869926,"data_compressed":1825202,"data_compressed_allocated":4841472,"data_compressed_original":63826196,"omap_allocated":0,"internal_metadata":0},"log_size":2589,"ondisk_log_size":2589,"up":64,"acting":64,"num_store_stats":7},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":155648,"data_stored":1771104,"data_compressed":27030,"data_compressed_allocated":110592,"data_compressed_original":1736704,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574858,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302984,"kb_used_data":1536,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068856,"statfs":{"total":96636764160,"available":96326508544,"internally_reserved":0,"allocated":1572864,"data_stored":7745270,"data_compressed":233576,"data_compressed_allocated":684032,"data_compressed_original":7273956,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51100000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39600000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]}]},{"osd":6,"up_from":13,"seq":55834574858,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303040,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068800,"statfs":{"total":96636764160,"available":96326451200,"internally_reserved":0,"allocated":1622016,"data_stored":10231388,"data_compressed":237941,"data_compressed_allocated":667648,"data_compressed_original":9654272,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.22}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]}]},{"osd":1,"up_from":0,"seq":55834574857,"num_pgs":0,"num_osds":0,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":0,"kb_used":0,"kb_used_data":0,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":0,"statfs":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},{"osd":0,"up_from":13,"seq":55834574858,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":302996,"kb_used_data":1548,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068844,"statfs":{"total":96636764160,"available":96326496256,"internally_reserved":0,"allocated":1585152,"data_stored":9580651,"data_compressed":264285,"data_compressed_allocated":643072,"data_compressed_original":9007736,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.438}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52600000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]},{"osd":2,"up_from":13,"seq":55834574857,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298540,"kb_used_data":1180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94073300,"statfs":{"total":96636764160,"available":96331059200,"internally_reserved":0,"allocated":1208320,"data_stored":5576817,"data_compressed":199145,"data_compressed_allocated":356352,"data_compressed_original":5046272,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.219}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29799999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45200000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.372}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303544,"kb_used_data":2068,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068296,"statfs":{"total":96636764160,"available":96325935104,"internally_reserved":0,"allocated":2117632,"data_stored":11906306,"data_compressed":326848,"data_compressed_allocated":876544,"data_compressed_original":11159772,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.254}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33500000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36199999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]}]},{"osd":4,"up_from":13,"seq":55834574858,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303856,"kb_used_data":2396,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94067984,"statfs":{"total":96636764160,"available":96325615616,"internally_reserved":0,"allocated":2453504,"data_stored":16493406,"data_compressed":412059,"data_compressed_allocated":1064960,"data_compressed_original":15581184,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.501}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42499999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574858,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298948,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072892,"statfs":{"total":96636764160,"available":96330641408,"internally_reserved":0,"allocated":1622016,"data_stored":7888449,"data_compressed":169339,"data_compressed_allocated":602112,"data_compressed_original":7263690,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.217}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.255}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28399999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40400000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1081344,"data_stored":8785700,"data_compressed":254583,"data_compressed_allocated":606208,"data_compressed_original":8417912,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":745472,"data_stored":5372234,"data_compressed":199145,"data_compressed_allocated":356352,"data_compressed_original":5046272,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":1654784,"data_stored":11701723,"data_compressed":326848,"data_compressed_allocated":876544,"data_compressed_original":11159772,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":1990656,"data_stored":16288823,"data_compressed":412059,"data_compressed_allocated":1064960,"data_compressed_original":15581184,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":1179648,"data_stored":7698600,"data_compressed":169418,"data_compressed_allocated":606208,"data_compressed_original":7271882,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":1257472,"data_stored":10072527,"data_compressed":238237,"data_compressed_allocated":684032,"data_compressed_original":9648658,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1052672,"data_stored":6950319,"data_compressed":224912,"data_compressed_allocated":647168,"data_compressed_original":6700516,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:07:54.484 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-24T18:07:54.485 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:07:54.485 INFO:tasks.thrashosds.thrasher:Adding osd 1 2024-04-24T18:07:54.485 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 1 2024-04-24T18:07:54.540 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:54.540 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:54.552 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:07:54.593 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:54.601+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:54.608 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:07:54.667 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:54.675+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:54.732 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #7, image image_client.0-clone7 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent8 2024-04-24T18:07:54.741 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:54.741 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:54.748 INFO:teuthology.orchestra.run.smithi049.stdout:1134 read 0xa20f8c thru 0xa22f2b (0x1fa0 bytes) 2024-04-24T18:07:54.750 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:54.754 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:07:54.761 INFO:teuthology.orchestra.run.smithi049.stdout:1136 trunc from 0x3025641 to 0xd03426e 2024-04-24T18:07:54.781 INFO:teuthology.orchestra.run.smithi049.stdout:1137 trunc from 0xd03426e to 0x4b81568 2024-04-24T18:07:54.781 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:54.789+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:54.803 INFO:teuthology.orchestra.run.smithi049.stdout:1138 write 0xe0cd75f thru 0xe0d3f3b (0x67dd bytes) 2024-04-24T18:07:54.809 INFO:teuthology.orchestra.run.smithi049.stdout:1139 punch from 0x4e09a0e to 0x4e18673, (0xec65 bytes) 2024-04-24T18:07:54.826 INFO:teuthology.orchestra.run.smithi049.stdout:1140 trunc from 0xe0d3f3c to 0x11ac414 2024-04-24T18:07:54.869 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:54.877+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:54.888 INFO:teuthology.orchestra.run.smithi049.stdout:1141 writesame 0x420fca6 thru 0x4212031 data_size 0x2(0x238c bytes) 2024-04-24T18:07:54.888 INFO:teuthology.orchestra.run.smithi049.stdout:1142 write 0x3ebc99a thru 0x3ec8f7e (0xc5e5 bytes) 2024-04-24T18:07:54.900 INFO:teuthology.orchestra.run.smithi049.stdout:1143 trunc from 0x4212032 to 0x6395a3b 2024-04-24T18:07:54.905 INFO:teuthology.orchestra.run.smithi049.stdout:1146 write 0x1e15730 thru 0x1e2065d (0xaf2e bytes) 2024-04-24T18:07:54.915 INFO:teuthology.orchestra.run.smithi049.stdout:1147 writesame 0x4ea45 thru 0x52332 data_size 0x15(0x38ee bytes) 2024-04-24T18:07:54.930 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-04-24T18:07:54.937+0000 7fa70e0ea700 -1 mon.a@0(leader).osd e44 definitely_dead 0 2024-04-24T18:07:54.944 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:54.945 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:54.959 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:07:54.962 INFO:teuthology.orchestra.run.smithi049.stdout:1148 writesame 0x19f783f thru 0x19fe60a data_size 0x2(0x6dcc bytes) 2024-04-24T18:07:54.973 INFO:teuthology.orchestra.run.smithi049.stdout:1150 read 0x3dd5bc0 thru 0x3de50cc (0xf50d bytes) 2024-04-24T18:07:54.973 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:54.981+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:1151 read 0x2dabdbb thru 0x2dbbbd9 (0xfe1f bytes) 2024-04-24T18:07:54.974 INFO:teuthology.orchestra.run.smithi049.stdout:1153 write 0x54db7d8 thru 0x54e0dcb (0x55f4 bytes) 2024-04-24T18:07:54.985 INFO:teuthology.orchestra.run.smithi049.stdout:1156 trunc from 0x6395a3b to 0x6d459f4 2024-04-24T18:07:54.990 INFO:teuthology.orchestra.run.smithi049.stdout:1158 punch from 0x49d067d to 0x49d08cc, (0x24f bytes) 2024-04-24T18:07:54.996 INFO:teuthology.orchestra.run.smithi049.stdout:1159 write 0x7ac75a0 thru 0x7ad0311 (0x8d72 bytes) 2024-04-24T18:07:55.001 INFO:teuthology.orchestra.run.smithi049.stdout:1160 read 0x133162d thru 0x1332545 (0xf19 bytes) 2024-04-24T18:07:55.001 INFO:teuthology.orchestra.run.smithi049.stdout:1161 punch from 0x72a6a6b to 0x72ad1d3, (0x6768 bytes) 2024-04-24T18:07:55.001 INFO:teuthology.orchestra.run.smithi049.stdout:1162 writesame 0x3490d90 thru 0x3494d12 data_size 0x1(0x3f83 bytes) 2024-04-24T18:07:55.010 INFO:teuthology.orchestra.run.smithi049.stdout:1163 read 0x772b79b thru 0x77336ba (0x7f20 bytes) 2024-04-24T18:07:55.015 INFO:teuthology.orchestra.run.smithi049.stdout:1164 trunc from 0x7ad0312 to 0xb560dd9 2024-04-24T18:07:55.020 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.020 INFO:teuthology.orchestra.run.smithi049.stdout:1166 writesame 0x92fb19a thru 0x9305f4d data_size 0x2(0xadb4 bytes) 2024-04-24T18:07:55.032 INFO:teuthology.orchestra.run.smithi049.stdout:1167 write 0x2a5410c thru 0x2a5d85b (0x9750 bytes) 2024-04-24T18:07:55.038 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.038 INFO:teuthology.orchestra.run.smithi049.stdout:1169 punch from 0xa694527 to 0xa699a61, (0x553a bytes) 2024-04-24T18:07:55.038 INFO:teuthology.orchestra.run.smithi049.stdout:1172 read 0x4310b3f thru 0x4313cf9 (0x31bb bytes) 2024-04-24T18:07:55.038 INFO:teuthology.orchestra.run.smithi049.stdout:1173 writesame 0x511e282 thru 0x5121c4d data_size 0x12(0x39cc bytes) 2024-04-24T18:07:55.044 INFO:teuthology.orchestra.run.smithi049.stdout:1174 trunc from 0xb560dd9 to 0x90ac318 2024-04-24T18:07:55.050 INFO:teuthology.orchestra.run.smithi049.stdout:1176 read 0x5a48228 thru 0x5a48557 (0x330 bytes) 2024-04-24T18:07:55.050 INFO:teuthology.orchestra.run.smithi049.stdout:1178 writesame 0x72c8cd thru 0x73c87c data_size 0x8(0xffb0 bytes) 2024-04-24T18:07:55.070 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:55.078+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:55.125 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:55.125 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:55.126 INFO:teuthology.orchestra.run.smithi049.stdout:1180 writesame 0x622859b thru 0x622f948 data_size 0x43(0x73ae bytes) 2024-04-24T18:07:55.138 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:07:55.139 INFO:teuthology.orchestra.run.smithi049.stdout:1181 write 0xae40f12 thru 0xae509dd (0xfacc bytes) 2024-04-24T18:07:55.144 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.144 INFO:teuthology.orchestra.run.smithi049.stdout:1183 read 0x62ea8eb thru 0x62f2aaf (0x81c5 bytes) 2024-04-24T18:07:55.145 INFO:teuthology.orchestra.run.smithi049.stdout:1184 punch from 0x936185d to 0x937184b, (0xffee bytes) 2024-04-24T18:07:55.155 INFO:teuthology.orchestra.run.smithi049.stdout:1186 trunc from 0xae509de to 0x20e54ea 2024-04-24T18:07:55.179 INFO:teuthology.orchestra.run.smithi049.stdout:1187 trunc from 0x20e54ea to 0xa716548 2024-04-24T18:07:55.181 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:55.190+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:55.184 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.184 INFO:teuthology.orchestra.run.smithi049.stdout:1189 read 0x17f6456 thru 0x17f821b (0x1dc6 bytes) 2024-04-24T18:07:55.184 INFO:teuthology.orchestra.run.smithi049.stdout:1190 punch from 0x1d62c27 to 0x1d632fe, (0x6d7 bytes) 2024-04-24T18:07:55.184 INFO:teuthology.orchestra.run.smithi049.stdout:1191 read 0x96e5e4b thru 0x96efa8a (0x9c40 bytes) 2024-04-24T18:07:55.184 INFO:teuthology.orchestra.run.smithi049.stdout:1195 write 0x991b9e6 thru 0x992296e (0x6f89 bytes) 2024-04-24T18:07:55.189 INFO:teuthology.orchestra.run.smithi049.stdout:1196 writesame 0x906475f thru 0x907210d data_size 0x13(0xd9af bytes) 2024-04-24T18:07:55.194 INFO:teuthology.orchestra.run.smithi049.stdout:1197 write 0x8f2194b thru 0x8f24047 (0x26fd bytes) 2024-04-24T18:07:55.198 INFO:teuthology.orchestra.run.smithi049.stdout:1199 write 0x5657259 thru 0x5660cd9 (0x9a81 bytes) 2024-04-24T18:07:55.203 INFO:teuthology.orchestra.run.smithi049.stdout:1200 write 0x34ae974 thru 0x34b04fb (0x1b88 bytes) 2024-04-24T18:07:55.206 INFO:teuthology.orchestra.run.smithi049.stdout:1201 writesame 0x614bfb1 thru 0x6159405 data_size 0x1(0xd455 bytes) 2024-04-24T18:07:55.221 INFO:teuthology.orchestra.run.smithi049.stdout:1203 read 0x91dcd5f thru 0x91ea66e (0xd910 bytes) 2024-04-24T18:07:55.227 INFO:teuthology.orchestra.run.smithi049.stdout:1204 writesame 0xb26aaa3 thru 0xb270458 data_size 0x1(0x59b6 bytes) 2024-04-24T18:07:55.234 INFO:teuthology.orchestra.run.smithi049.stdout:1206 write 0xc292819 thru 0xc2a225d (0xfa45 bytes) 2024-04-24T18:07:55.243 INFO:teuthology.orchestra.run.smithi049.stdout:1207 write 0x62d54c2 thru 0x62d9087 (0x3bc6 bytes) 2024-04-24T18:07:55.252 INFO:teuthology.orchestra.run.smithi049.stdout:1208 write 0xd272e0c thru 0xd27719d (0x4392 bytes) 2024-04-24T18:07:55.256 INFO:teuthology.orchestra.run.smithi049.stdout:1209 trunc from 0xd27719e to 0xc6fda5d 2024-04-24T18:07:55.265 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.265 INFO:teuthology.orchestra.run.smithi049.stdout:1212 punch from 0xba30fdd to 0xba3888e, (0x78b1 bytes) 2024-04-24T18:07:55.265 INFO:teuthology.orchestra.run.smithi049.stdout:1213 read 0x7ef41ef thru 0x7f030d2 (0xeee4 bytes) 2024-04-24T18:07:55.265 INFO:teuthology.orchestra.run.smithi049.stdout:1214 write 0x9c387a6 thru 0x9c3c0ed (0x3948 bytes) 2024-04-24T18:07:55.271 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:55.279+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:55.273 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:55.274 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:55.275 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.275 INFO:teuthology.orchestra.run.smithi049.stdout:1219 read 0x426b711 thru 0x42735b3 (0x7ea3 bytes) 2024-04-24T18:07:55.275 INFO:teuthology.orchestra.run.smithi049.stdout:1221 writesame 0x8239fd6 thru 0x8244796 data_size 0x7(0xa7c1 bytes) 2024-04-24T18:07:55.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:07:55.295 INFO:teuthology.orchestra.run.smithi049.stdout:1222 trunc from 0xc6fda5d to 0x3e09e5e 2024-04-24T18:07:55.311 INFO:teuthology.orchestra.run.smithi049.stdout:1223 read 0x27576c7 thru 0x27621e9 (0xab23 bytes) 2024-04-24T18:07:55.325 INFO:teuthology.orchestra.run.smithi049.stdout:1224 write 0xc13b606 thru 0xc140a78 (0x5473 bytes) 2024-04-24T18:07:55.339 INFO:teuthology.orchestra.run.smithi049.stdout:1225 trunc from 0xc140a79 to 0x65ee3ce 2024-04-24T18:07:55.348 INFO:teuthology.orchestra.run.smithi049.stdout:1226 read 0x282874 thru 0x286774 (0x3f01 bytes) 2024-04-24T18:07:55.356 INFO:teuthology.orchestra.run.smithi049.stdout:1228 write 0x887164e thru 0x887dfb6 (0xc969 bytes) 2024-04-24T18:07:55.364 INFO:teuthology.orchestra.run.smithi049.stdout:1229 read 0x8ca5e8 thru 0x8cd8e6 (0x32ff bytes) 2024-04-24T18:07:55.364 INFO:teuthology.orchestra.run.smithi049.stdout:1232 read 0x336ab0a thru 0x336db5a (0x3051 bytes) 2024-04-24T18:07:55.365 INFO:teuthology.orchestra.run.smithi049.stdout:1234 trunc from 0x887dfb7 to 0x7d932ab 2024-04-24T18:07:55.372 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:55.380+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:55.379 INFO:teuthology.orchestra.run.smithi049.stdout:1236 trunc from 0x7d932ab to 0x37b4edb 2024-04-24T18:07:55.391 INFO:teuthology.orchestra.run.smithi049.stdout:1237 writesame 0x471222e thru 0x471380b data_size 0x12(0x15de bytes) 2024-04-24T18:07:55.391 INFO:teuthology.orchestra.run.smithi049.stdout:1238 punch from 0x1d2abbc to 0x1d2cd93, (0x21d7 bytes) 2024-04-24T18:07:55.391 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.391 INFO:teuthology.orchestra.run.smithi049.stdout:1240 read 0x2498fd0 thru 0x24a0460 (0x7491 bytes) 2024-04-24T18:07:55.401 INFO:teuthology.orchestra.run.smithi049.stdout:1241 write 0x99b6de9 thru 0x99b9611 (0x2829 bytes) 2024-04-24T18:07:55.405 INFO:teuthology.orchestra.run.smithi049.stdout:1242 write 0xad4796 thru 0xaddaa0 (0x930b bytes) 2024-04-24T18:07:55.458 INFO:teuthology.orchestra.run.smithi049.stdout:1243 writesame 0x1970b90 thru 0x197836f data_size 0x38(0x77e0 bytes) 2024-04-24T18:07:55.459 INFO:teuthology.orchestra.run.smithi049.stdout:truncating to largest ever: 0xeba6d63 2024-04-24T18:07:55.465 INFO:teuthology.orchestra.run.smithi049.stdout:1244 trunc from 0x99b9612 to 0xeba6d63 2024-04-24T18:07:55.470 INFO:teuthology.orchestra.run.smithi049.stdout:1246 writesame 0x4c6b8a6 thru 0x4c79fb4 data_size 0x1(0xe70f bytes) 2024-04-24T18:07:55.471 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:55.471 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:55.474 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:55.482+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:55.484 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:07:55.484 INFO:teuthology.orchestra.run.smithi049.stdout:1247 trunc from 0xeba6d63 to 0xe2a814b 2024-04-24T18:07:55.489 INFO:teuthology.orchestra.run.smithi049.stdout:1248 punch from 0xd07fd1e to 0xd08fbdf, (0xfec1 bytes) 2024-04-24T18:07:55.489 INFO:teuthology.orchestra.run.smithi049.stdout:1252 trunc from 0xe2a814b to 0xe7ed4b7 2024-04-24T18:07:55.494 INFO:teuthology.orchestra.run.smithi049.stdout:1253 write 0x1404448 thru 0x140eb77 (0xa730 bytes) 2024-04-24T18:07:55.499 INFO:teuthology.orchestra.run.smithi049.stdout:1255 write 0x5833b46 thru 0x583c263 (0x871e bytes) 2024-04-24T18:07:55.504 INFO:teuthology.orchestra.run.smithi049.stdout:1256 writesame 0xb857668 thru 0xb857c3d data_size 0x9(0x5d6 bytes) 2024-04-24T18:07:55.506 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:55.509 INFO:teuthology.orchestra.run.smithi049.stdout:1259 trunc from 0xe7ed4b7 to 0xc12a0a3 2024-04-24T18:07:55.515 INFO:teuthology.orchestra.run.smithi049.stdout:1261 clone 10 order 20 su 65536 sc 7 2024-04-24T18:07:55.575 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:55.583+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:55.618 INFO:teuthology.orchestra.run.smithi049.stderr:marked in osd.1. 2024-04-24T18:07:55.629 INFO:tasks.thrashosds.thrasher:Added osd 1 2024-04-24T18:07:55.652 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:55.653 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:55.663 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:07:55.676 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:55.684+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:55.776 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:55.785+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:55.788 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:55.788 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:55.798 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:07:55.881 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:55.890+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:55.961 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:55.961 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:55.971 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:07:55.978 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:55.986+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:56.079 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:56.087+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:56.136 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:56.136 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:56.146 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:07:56.188 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:56.196+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:56.283 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:56.290+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:56.311 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:56.312 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:56.322 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:07:56.383 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:56.392+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:56.422 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:07:56.446 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:56.446 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:56.456 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:07:56.497 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:56.505+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:56.576 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:56.588 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:56.596+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:56.594 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:56.622 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:56.624 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:07:56.625 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:56.625 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:56.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:07:56.654 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:56.680 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:56.689 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:56.698+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:56.706 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:56.735 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:56.760 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:56.761 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:56.761 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:56.761 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:56.762 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:56.762 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:56.762 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:56.762 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:56.762 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:56.769 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:56.769 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:56.769 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:07:56.770 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:56.780 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:07:56.819 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:56.827+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:56.894 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:56.902+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:56.943 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:56.943 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:07:56.943 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:07:56.943 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:07:56.943 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:56.953 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:07:56.995 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:57.003+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:57.078 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:07:57.078 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:07:57.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:07:57.078 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468806121999997, 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:07:57.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:07:57.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:07:57.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.084 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468737070000003, 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:57.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468712654999997, 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.090 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:07:57.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:07:57.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.095 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:57.097 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468689505999997, 2024-04-24T18:07:57.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:07:57.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.102 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468608938000003, 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468584128000003, 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.108 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:07:57.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:57.114 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468564016000002, 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468493353, 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.468481580000002, 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.130 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.048159751, 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:07:57.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:07:57.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.136 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.048112568999997, 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:57.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.142 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.048087447999997, 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.143 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:07:57.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:07:57.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.048035314000003, 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:07:57.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.047971850000003, 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.154 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:07:57.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:07:57.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:07:57.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.536801551000003, 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073643390000000001, 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.160 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 156 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548239+0000", 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8899999999999996e-06 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548246+0000", 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9370000000000004e-06 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548259+0000", 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549586+0000", 2024-04-24T18:07:57.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001326743 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549641+0000", 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5856999999999997e-05 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549713+0000", 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1970000000000004e-05 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550899+0000", 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001185567 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555532+0000", 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0046325760000000002 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555573+0000", 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1730000000000002e-05 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555595+0000", 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1324000000000001e-05 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.536766178000001, 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.166 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.536741151000001, 2024-04-24T18:07:57.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.176 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.536661914, 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:07:57.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:07:57.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:07:57.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.2744731329999999, 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:07:57.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:07:57.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:07:57.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:07:57.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.2733670640000003, 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:07:57.186 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:07:57.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:07:57.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:07:57.190 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:07:57.191 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:07:57.192 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:57.199+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:57.292 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:57.300+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:57.321 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:57.321 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:57.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.321 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4.ef6cece6 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:07:57.321 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:07:57.321 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.71223995600000001, 2024-04-24T18:07:57.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.71226018999999996, 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "waiting for peered", 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 1 2024-04-24T18:07:57.325 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:57.331 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:07:57.415 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:57.423+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:57.494 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:57.502+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:57.500 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:57.500 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:57.500 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:57.500 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:57.500 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:57.510 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:07:57.595 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:57.603+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:57.617 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:07:57.686 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone9 from 0xc12a0a3 (overlap 0x11ac414) to 0x1c46ce 2024-04-24T18:07:57.696 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:57.704+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:57.714 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:57.714 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:57.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:57.714 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31810 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 2920448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.025725735, 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052993206000000001, 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31810 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691739+0000", 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691742+0000", 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2800000000000002e-06 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691747+0000", 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4319999999999998e-06 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691750+0000", 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.247e-06 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744520+0000", 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052769498999999997 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744623+0000", 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000102808 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744733+0000", 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110282 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.0121409750000003, 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.719 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:07:57.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32297 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3233792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9997668920000002, 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052981070999999998, 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32297 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717698+0000", 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717705+0000", 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5529999999999998e-06 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717711+0000", 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3849999999999999e-06 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717715+0000", 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770506+0000", 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052791267000000003 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770587+0000", 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0783000000000002e-05 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770680+0000", 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2497000000000006e-05 2024-04-24T18:07:57.728 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32299 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9997458889999997, 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053161606, 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.729 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32299 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717719+0000", 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717722+0000", 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.43e-06 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717729+0000", 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.576e-06 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717733+0000", 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0119999999999999e-06 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770707+0000", 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052974083999999998 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770791+0000", 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3418000000000007e-05 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770881+0000", 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0490999999999998e-05 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32301 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.999728019, 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053336189999999999, 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.733 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32301 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717737+0000", 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717740+0000", 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4090000000000001e-06 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717746+0000", 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9989999999999997e-06 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717752+0000", 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0349999999999998e-06 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770915+0000", 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053162938 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770989+0000", 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4567999999999995e-05 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771074+0000", 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4598000000000006e-05 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32303 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3270656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9996149409999999, 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053426976000000001, 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.737 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32303 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:07:57.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717850+0000", 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717854+0000", 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0509999999999998e-06 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717863+0000", 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.346e-06 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717877+0000", 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3559e-05 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771102+0000", 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053224869000000001 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771184+0000", 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2564000000000004e-05 2024-04-24T18:07:57.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771278+0000", 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3178999999999997e-05 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32305 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3278848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9995827730000002, 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053587411000000001, 2024-04-24T18:07:57.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32305 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717882+0000", 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717885+0000", 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8679999999999999e-06 2024-04-24T18:07:57.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717892+0000", 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717896+0000", 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9360000000000001e-06 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771305+0000", 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053409310000000002 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771373+0000", 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8417999999999994e-05 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771470+0000", 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6827e-05 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32307 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3289088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.745 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9995657319999998, 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053767057, 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32307 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:07:57.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717899+0000", 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717902+0000", 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5019999999999999e-06 2024-04-24T18:07:57.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717908+0000", 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7699999999999998e-06 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717911+0000", 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4979999999999998e-06 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771498+0000", 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053586747999999997 2024-04-24T18:07:57.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771571+0000", 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3168000000000001e-05 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771667+0000", 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5810000000000006e-05 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.749 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32311 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3307520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9982693469999999, 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053356705999999997, 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32311 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:57.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:07:57.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719199+0000", 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.88e-06 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719205+0000", 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719209+0000", 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0749999999999999e-06 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772008+0000", 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052798572000000002 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772048+0000", 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0238999999999999e-05 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772553+0000", 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050474099999999996 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32313 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3315712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9982512889999997, 2024-04-24T18:07:57.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053105309000000003, 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32313 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719216+0000", 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719223+0000", 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3010000000000003e-06 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719226+0000", 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7610000000000001e-06 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772130+0000", 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052903238999999998 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772209+0000", 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9486000000000004e-05 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772320+0000", 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011052099999999999 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32315 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3334144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9982346460000002, 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053250991999999997, 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32315 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719233+0000", 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1789999999999998e-06 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719240+0000", 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5689999999999999e-06 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719244+0000", 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5870000000000001e-06 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772345+0000", 2024-04-24T18:07:57.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053100972000000003 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772413+0000", 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8071000000000006e-05 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772482+0000", 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8950999999999998e-05 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32317 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9981525849999997, 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053370316000000001, 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32317 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:07:57.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719312+0000", 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719316+0000", 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9400000000000004e-06 2024-04-24T18:07:57.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719326+0000", 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2439999999999999e-06 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719331+0000", 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7049999999999996e-06 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772583+0000", 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053251860999999998 2024-04-24T18:07:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772628+0000", 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5289e-05 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772683+0000", 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4882000000000001e-05 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32319 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3352576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9981292420000001, 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053459060000000003, 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32319 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719338+0000", 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5490000000000001e-06 2024-04-24T18:07:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719346+0000", 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5059999999999998e-06 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719350+0000", 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1999999999999996e-06 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772709+0000", 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053359182999999998 2024-04-24T18:07:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772748+0000", 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8654e-05 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772795+0000", 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7437e-05 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32321 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3360768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9981110319999997, 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053627222000000002, 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32321 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:07:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719354+0000", 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719361+0000", 2024-04-24T18:07:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7449999999999998e-06 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719367+0000", 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3940000000000001e-06 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719378+0000", 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1204e-05 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772830+0000", 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053451920999999999 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772896+0000", 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5623000000000005e-05 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772982+0000", 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5729000000000005e-05 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32327 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3389440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9970948240000004, 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053038750000000003, 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32327 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720370+0000", 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720373+0000", 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5830000000000001e-06 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720380+0000", 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1550000000000001e-06 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720384+0000", 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0260000000000001e-06 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773322+0000", 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052937495000000001 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773364+0000", 2024-04-24T18:07:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2561999999999997e-05 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773409+0000", 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5333000000000001e-05 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32329 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3397632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9970771249999997, 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053169842000000002, 2024-04-24T18:07:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32329 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720391+0000", 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7779999999999999e-06 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720396+0000", 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6729999999999997e-06 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720400+0000", 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7179999999999998e-06 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773435+0000", 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053035169 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773497+0000", 2024-04-24T18:07:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1715000000000001e-05 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773558+0000", 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1171000000000001e-05 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32331 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3416064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9970609960000001, 2024-04-24T18:07:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053284320000000003, 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32331 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720404+0000", 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720406+0000", 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3190000000000002e-06 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720412+0000", 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8359999999999997e-06 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720416+0000", 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000004e-06 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773583+0000", 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053167076000000001 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773638+0000", 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105999999999997e-05 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773689+0000", 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0521e-05 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32333 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3424256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9970433840000004, 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053395479000000003, 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32333 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720424+0000", 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720430+0000", 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720434+0000", 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8210000000000003e-06 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773721+0000", 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053287297999999997 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773762+0000", 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1245000000000002e-05 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773818+0000", 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5585999999999999e-05 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32388 4.3 4:c61fab1f:::rbd_object_map.117432c990a0.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9949411550000002, 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.055543581000000002, 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32388 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722527+0000", 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0319999999999999e-06 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722535+0000", 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9170000000000006e-06 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722539+0000", 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9709999999999998e-06 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774649+0000", 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052110109000000002 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774719+0000", 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9562999999999998e-05 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775050+0000", 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000331078 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777149+0000", 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020996639999999998 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777996+0000", 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084650899999999996 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778027+0000", 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0818999999999997e-05 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778068+0000", 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1338000000000003e-05 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 1.099666169, 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.801 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:57.803 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:07:57.803 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:57.810+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:57.812 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #8, image image_client.0-clone8 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent9 2024-04-24T18:07:57.872 INFO:teuthology.orchestra.run.smithi049.stdout:1262 punch from 0x3757ed2 to 0x3765580, (0xd6ae bytes) 2024-04-24T18:07:57.875 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:57.876 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:57.876 INFO:teuthology.orchestra.run.smithi049.stdout:1266 read 0x7808132 thru 0x780c4f6 (0x43c5 bytes) 2024-04-24T18:07:57.876 INFO:teuthology.orchestra.run.smithi049.stdout:1267 writesame 0x8f93d0d thru 0x8f9773e data_size 0x27(0x3a32 bytes) 2024-04-24T18:07:57.879 INFO:teuthology.orchestra.run.smithi049.stdout:1269 trunc from 0xc12a0a3 to 0xcf6806e 2024-04-24T18:07:57.886 INFO:teuthology.orchestra.run.smithi049.stdout:1270 trunc from 0xcf6806e to 0xbb4b6a4 2024-04-24T18:07:57.902 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:57.902 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:57.903 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:57.910+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:57.905 INFO:teuthology.orchestra.run.smithi049.stdout:1275 trunc from 0xbb4b6a4 to 0xe2a7db2 2024-04-24T18:07:57.910 INFO:teuthology.orchestra.run.smithi049.stdout:1277 write 0x582e782 thru 0x583445f (0x5cde bytes) 2024-04-24T18:07:57.919 INFO:teuthology.orchestra.run.smithi049.stdout:1278 write 0xcb877f7 thru 0xcb889c7 (0x11d1 bytes) 2024-04-24T18:07:57.929 INFO:teuthology.orchestra.run.smithi049.stdout:1279 trunc from 0xe2a7db2 to 0x6c1236d 2024-04-24T18:07:57.953 INFO:teuthology.orchestra.run.smithi049.stdout:1281 trunc from 0x6c1236d to 0x4f28626 2024-04-24T18:07:57.973 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41535 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975161+0000", 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0027945449999999998, 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028169919999999999, 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41535 2024-04-24T18:07:57.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975161+0000", 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975161+0000", 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975160+0000", 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975163+0000", 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3499999999999999e-06 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975167+0000", 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8789999999999996e-06 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975169+0000", 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1830000000000001e-06 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41536 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975172+0000", 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002783682, 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0029043770000000001, 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41536 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975172+0000", 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975172+0000", 2024-04-24T18:07:57.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975171+0000", 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975173+0000", 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.437e-06 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975176+0000", 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.937e-06 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975178+0000", 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9659999999999999e-06 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41558 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975180+0000", 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0027748719999999998, 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0029697719999999999, 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.980 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41558 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975180+0000", 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975180+0000", 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975180+0000", 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975181+0000", 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2359999999999999e-06 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975185+0000", 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0850000000000001e-06 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975186+0000", 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.863e-06 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41559 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.983 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975189+0000", 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0027666240000000001, 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0030458099999999999, 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41559 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975189+0000", 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975189+0000", 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975189+0000", 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975190+0000", 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.282e-06 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975193+0000", 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.886e-06 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975195+0000", 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9819999999999998e-06 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41560 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975197+0000", 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0027585230000000001, 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031043960000000002, 2024-04-24T18:07:57.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41560 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975197+0000", 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975197+0000", 2024-04-24T18:07:57.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975197+0000", 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975198+0000", 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2410000000000001e-06 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975201+0000", 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9069999999999999e-06 2024-04-24T18:07:57.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975203+0000", 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.962e-06 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41561 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975205+0000", 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0027503139999999998, 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031664969999999999, 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.989 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41561 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975205+0000", 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975205+0000", 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975205+0000", 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975209+0000", 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3880000000000002e-06 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975212+0000", 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1410000000000001e-06 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975214+0000", 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.902e-06 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41562 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975217+0000", 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002738744, 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0032208760000000001, 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.992 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41562 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975217+0000", 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975217+0000", 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975216+0000", 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975218+0000", 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.308e-06 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975221+0000", 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975223+0000", 2024-04-24T18:07:57.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9800000000000001e-06 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41563 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975231+0000", 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0027247999999999999, 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0032716300000000002, 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41563 2024-04-24T18:07:57.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975231+0000", 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975231+0000", 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975230+0000", 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:57.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975232+0000", 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4309999999999999e-06 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975236+0000", 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.741e-06 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975240+0000", 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.281e-06 2024-04-24T18:07:57.997 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41564 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975244+0000", 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00271134, 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033290749999999999, 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41564 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.998 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975244+0000", 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975244+0000", 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975244+0000", 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:57.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975248+0000", 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1559999999999997e-06 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975256+0000", 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3750000000000003e-06 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975260+0000", 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.895e-06 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.000 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41586 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975265+0000", 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0026903109999999999, 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033793429999999999, 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41586 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975265+0000", 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975265+0000", 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975265+0000", 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975267+0000", 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.593e-06 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975275+0000", 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4950000000000002e-06 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975277+0000", 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4700000000000001e-06 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.003 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41587 4.f 4:f7730267:::rbd_object_map.1174578192f5:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975280+0000", 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002675345, 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003437847, 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "waiting for sub ops", 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41587 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975280+0000", 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975280+0000", 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975280+0000", 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975282+0000", 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3199999999999998e-06 2024-04-24T18:07:58.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975285+0000", 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3359999999999999e-06 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975287+0000", 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9420000000000002e-06 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975329+0000", 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:07:58.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975412+0000", 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3832999999999995e-05 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975774+0000", 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00036178799999999999 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.976531+0000", 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00075686199999999996 2024-04-24T18:07:58.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41588 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975294+0000", 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002661666, 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035198540000000002, 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41588 2024-04-24T18:07:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975294+0000", 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975294+0000", 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975293+0000", 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975296+0000", 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.249e-06 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975301+0000", 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975304+0000", 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.010 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41589 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975307+0000", 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0026483700000000002, 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035742159999999999, 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41589 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975307+0000", 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975307+0000", 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975307+0000", 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975309+0000", 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975314+0000", 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2390000000000002e-06 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975318+0000", 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8639999999999998e-06 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.013 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41590 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975322+0000", 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0026329309999999998, 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036520849999999998, 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41590 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975322+0000", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975322+0000", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975322+0000", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975324+0000", 2024-04-24T18:07:58.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975330+0000", 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6890000000000001e-06 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975332+0000", 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41591 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.016 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975335+0000", 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0026204359999999999, 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037092649999999998, 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41591 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975335+0000", 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975335+0000", 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975335+0000", 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975336+0000", 2024-04-24T18:07:58.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.403e-06 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975339+0000", 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9969999999999999e-06 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975341+0000", 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0080000000000001e-06 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41592 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975344+0000", 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002611721, 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037421300000000002, 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41592 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975344+0000", 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975344+0000", 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975343+0000", 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975345+0000", 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.37e-06 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975349+0000", 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3329999999999998e-06 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975351+0000", 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9860000000000001e-06 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41593 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975353+0000", 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002601992, 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0037814419999999999, 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41593 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975353+0000", 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975353+0000", 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975353+0000", 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975354+0000", 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.302e-06 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975357+0000", 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0089999999999999e-06 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975360+0000", 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3970000000000001e-06 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41613 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975363+0000", 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0025927379999999998, 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038518960000000001, 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41613 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975363+0000", 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975363+0000", 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975362+0000", 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975368+0000", 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6250000000000004e-06 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975374+0000", 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1070000000000001e-06 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975378+0000", 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41614 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975382+0000", 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0025731719999999999, 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038971589999999999, 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41614 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975382+0000", 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975382+0000", 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975382+0000", 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975384+0000", 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2479999999999999e-06 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975398+0000", 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3869000000000001e-05 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975403+0000", 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2089999999999996e-06 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.031 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41615 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975408+0000", 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0025469580000000002, 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039375850000000004, 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41615 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.032 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975408+0000", 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975408+0000", 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975408+0000", 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975411+0000", 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7700000000000002e-06 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975417+0000", 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4309999999999999e-06 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975421+0000", 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0249999999999996e-06 2024-04-24T18:07:58.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41616 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975425+0000", 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0025298780000000002, 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0039875400000000004, 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41616 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.035 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975425+0000", 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975425+0000", 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975425+0000", 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975427+0000", 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.351e-06 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975432+0000", 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3510000000000004e-06 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975434+0000", 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2189999999999998e-06 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41617 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975437+0000", 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0025188110000000001, 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040302289999999998, 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41617 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975437+0000", 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975437+0000", 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975436+0000", 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975438+0000", 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.46e-06 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975441+0000", 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9689999999999999e-06 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975443+0000", 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9429999999999999e-06 2024-04-24T18:07:58.040 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41618 4.4 4.27fdb104 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.975445+0000", 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0025103439999999999, 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0040804099999999996, 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41618 2024-04-24T18:07:58.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975445+0000", 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975445+0000", 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975445+0000", 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975446+0000", 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.145e-06 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975449+0000", 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9019999999999999e-06 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.975451+0000", 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8029999999999999e-06 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41625 4.f 4:f7730267:::rbd_object_map.1174578192f5:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.977602+0000", 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00035287699999999998, 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019973669999999999, 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "waiting for sub ops", 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41625 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977602+0000", 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977602+0000", 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977601+0000", 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977609+0000", 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7009999999999996e-06 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977623+0000", 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3821e-05 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977629+0000", 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2430000000000002e-06 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977724+0000", 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5029000000000001e-05 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977791+0000", 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6670999999999999e-05 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.978179+0000", 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00038803699999999998 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41626 4.f 4:f7730267:::rbd_object_map.1174578192f5:head [call lock.assert_locked in=44b,call rbd.object_map_update in=38b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.977773+0000", 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00018220700000000001, 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019207499999999999, 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "waiting for sub ops", 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41626 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977773+0000", 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977773+0000", 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977772+0000", 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977778+0000", 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1489999999999999e-06 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977788+0000", 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.9170000000000001e-06 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.977795+0000", 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5529999999999998e-06 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.978563+0000", 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00076871600000000002 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.978661+0000", 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7607000000000002e-05 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.979061+0000", 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00040030699999999997 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:41627 4.f 4.e640ceef (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.979701+0000", 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9982543, 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4585000000000006e-05, 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.052 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 41627 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.979701+0000", 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.979701+0000", 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.979700+0000", 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.979706+0000", 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1689999999999997e-06 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.979728+0000", 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1387e-05 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.979737+0000", 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0410000000000003e-06 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:07:58.055 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 26 2024-04-24T18:07:58.056 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:58.057 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:07:58.059 INFO:teuthology.orchestra.run.smithi049.stdout:1284 punch from 0xd3ab9d to 0xd3f100, (0x4563 bytes) 2024-04-24T18:07:58.059 INFO:teuthology.orchestra.run.smithi049.stdout:1286 read 0x470fdbd thru 0x4713cb8 (0x3efc bytes) 2024-04-24T18:07:58.059 INFO:teuthology.orchestra.run.smithi049.stdout:1290 writesame 0x210e38a thru 0x211264e data_size 0x1(0x42c5 bytes) 2024-04-24T18:07:58.059 INFO:teuthology.orchestra.run.smithi049.stdout:1291 trunc from 0x4f28626 to 0xbd88b52 2024-04-24T18:07:58.059 INFO:teuthology.orchestra.run.smithi049.stdout:1293 write 0xf37537 thru 0xf454cb (0xdf95 bytes) 2024-04-24T18:07:58.059 INFO:teuthology.orchestra.run.smithi049.stdout:1294 read 0x832c956 thru 0x8337415 (0xaac0 bytes) 2024-04-24T18:07:58.059 INFO:teuthology.orchestra.run.smithi049.stdout:1296 punch from 0x51b475a to 0x51ba930, (0x61d6 bytes) 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1298 read 0x381f013 thru 0x382afcd (0xbfbb bytes) 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1302 read 0x2756e23 thru 0x276575d (0xe93b bytes) 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1303 read 0xb670cf9 thru 0xb67f038 (0xe340 bytes) 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1304 write 0x1802e6c thru 0x180345b (0x5f0 bytes) 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1307 read 0xa3a30dc thru 0xa3ac798 (0x96bd bytes) 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1308 writesame 0x6a19e1e thru 0x6a27df8 data_size 0x1(0xdfdb bytes) 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1309 trunc from 0xbd88b52 to 0xaaf06df 2024-04-24T18:07:58.060 INFO:teuthology.orchestra.run.smithi049.stdout:1310 writesame 0x2d05fb8 thru 0x2d145a2 data_size 0x1(0xe5eb bytes) 2024-04-24T18:07:58.061 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:58.065+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:58.077 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.077 INFO:teuthology.orchestra.run.smithi049.stdout:1314 punch from 0x4648b4d to 0x4650c05, (0x80b8 bytes) 2024-04-24T18:07:58.077 INFO:teuthology.orchestra.run.smithi049.stdout:1315 read 0x55d4955 thru 0x55d8d37 (0x43e3 bytes) 2024-04-24T18:07:58.077 INFO:teuthology.orchestra.run.smithi049.stdout:1317 writesame 0x21a6e32 thru 0x21b2a69 data_size 0x4c(0xbc38 bytes) 2024-04-24T18:07:58.084 INFO:teuthology.orchestra.run.smithi049.stdout:1319 read 0xaa6586a thru 0xaa6f238 (0x99cf bytes) 2024-04-24T18:07:58.091 INFO:teuthology.orchestra.run.smithi049.stdout:1320 trunc from 0xaaf06df to 0x4cac0eb 2024-04-24T18:07:58.111 INFO:teuthology.orchestra.run.smithi049.stdout:1322 read 0x34965c0 thru 0x349f86b (0x92ac bytes) 2024-04-24T18:07:58.112 INFO:teuthology.orchestra.run.smithi049.stdout:1325 read 0x21dbe64 thru 0x21dc1f2 (0x38f bytes) 2024-04-24T18:07:58.125 INFO:teuthology.orchestra.run.smithi049.stdout:1326 writesame 0x9cd6434 thru 0x9ce3000 data_size 0x63(0xcbcd bytes) 2024-04-24T18:07:58.131 INFO:teuthology.orchestra.run.smithi049.stdout:1328 writesame 0xb731774 thru 0xb73930e data_size 0x1(0x7b9b bytes) 2024-04-24T18:07:58.131 INFO:teuthology.orchestra.run.smithi049.stdout:1329 read 0x204a630 thru 0x204adf1 (0x7c2 bytes) 2024-04-24T18:07:58.131 INFO:teuthology.orchestra.run.smithi049.stdout:1330 punch from 0xb689258 to 0xb68eb74, (0x591c bytes) 2024-04-24T18:07:58.131 INFO:teuthology.orchestra.run.smithi049.stdout:1333 write 0xb0b1340 thru 0xb0b72af (0x5f70 bytes) 2024-04-24T18:07:58.134 INFO:teuthology.orchestra.run.smithi049.stdout:1335 punch from 0xaf89cee to 0xaf967a8, (0xcaba bytes) 2024-04-24T18:07:58.134 INFO:teuthology.orchestra.run.smithi049.stdout:1336 read 0x5d568dc thru 0x5d64ecd (0xe5f2 bytes) 2024-04-24T18:07:58.134 INFO:teuthology.orchestra.run.smithi049.stdout:1340 read 0x8bfc4cf thru 0x8c00df8 (0x492a bytes) 2024-04-24T18:07:58.134 INFO:teuthology.orchestra.run.smithi049.stdout:1341 read 0x999e1a7 thru 0x99a0b0b (0x2965 bytes) 2024-04-24T18:07:58.134 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.135 INFO:teuthology.orchestra.run.smithi049.stdout:1343 write 0x58e0c49 thru 0x58e37a4 (0x2b5c bytes) 2024-04-24T18:07:58.136 INFO:teuthology.orchestra.run.smithi049.stdout:1344 read 0x56c8043 thru 0x56d408f (0xc04d bytes) 2024-04-24T18:07:58.136 INFO:teuthology.orchestra.run.smithi049.stdout:1345 write 0x293e32e thru 0x294d51b (0xf1ee bytes) 2024-04-24T18:07:58.150 INFO:teuthology.orchestra.run.smithi049.stdout:1346 trunc from 0xb73930f to 0x2626112 2024-04-24T18:07:58.158 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:58.166+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:58.168 INFO:teuthology.orchestra.run.smithi049.stdout:1347 read 0x20db049 thru 0x20e3ff3 (0x8fab bytes) 2024-04-24T18:07:58.168 INFO:teuthology.orchestra.run.smithi049.stdout:1350 read 0x1ba2c45 thru 0x1ba6a26 (0x3de2 bytes) 2024-04-24T18:07:58.174 INFO:teuthology.orchestra.run.smithi049.stdout:1351 writesame 0x3335139 thru 0x33443b3 data_size 0x41(0xf27b bytes) 2024-04-24T18:07:58.174 INFO:teuthology.orchestra.run.smithi049.stdout:1352 read 0x206f21 thru 0x21181e (0xa8fe bytes) 2024-04-24T18:07:58.175 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.192 INFO:teuthology.orchestra.run.smithi049.stdout:1356 write 0xdc180c1 thru 0xdc22b15 (0xaa55 bytes) 2024-04-24T18:07:58.197 INFO:teuthology.orchestra.run.smithi049.stdout:1357 trunc from 0xdc22b16 to 0xc6262fa 2024-04-24T18:07:58.206 INFO:teuthology.orchestra.run.smithi049.stdout:1358 write 0x62e011d thru 0x62e6ec4 (0x6da8 bytes) 2024-04-24T18:07:58.215 INFO:teuthology.orchestra.run.smithi049.stdout:1359 write 0xd624217 thru 0xd62aed8 (0x6cc2 bytes) 2024-04-24T18:07:58.232 INFO:teuthology.orchestra.run.smithi049.stdout:1361 trunc from 0xd62aed9 to 0xfe2a1a 2024-04-24T18:07:58.243 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:58.243 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:58.243 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:58.243 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:58.243 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:58.259 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:58.267+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:58.260 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:07:58.268 INFO:teuthology.orchestra.run.smithi049.stdout:1362 read 0x4650d2 thru 0x46b8b8 (0x67e7 bytes) 2024-04-24T18:07:58.278 INFO:teuthology.orchestra.run.smithi049.stdout:1364 write 0x4c49817 thru 0x4c550a5 (0xb88f bytes) 2024-04-24T18:07:58.282 INFO:teuthology.orchestra.run.smithi049.stdout:1366 read 0x3401c85 thru 0x34051dc (0x3558 bytes) 2024-04-24T18:07:58.282 INFO:teuthology.orchestra.run.smithi049.stdout:1367 write 0x2985163 thru 0x2993197 (0xe035 bytes) 2024-04-24T18:07:58.285 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.286 INFO:teuthology.orchestra.run.smithi049.stdout:1369 read 0x25cd0dd thru 0x25da123 (0xd047 bytes) 2024-04-24T18:07:58.302 INFO:teuthology.orchestra.run.smithi049.stdout:1370 write 0xe3d7c5c thru 0xe3e2907 (0xacac bytes) 2024-04-24T18:07:58.319 INFO:teuthology.orchestra.run.smithi049.stdout:1371 trunc from 0xe3e2908 to 0x2fe0625 2024-04-24T18:07:58.335 INFO:teuthology.orchestra.run.smithi049.stdout:1372 write 0x3f508f8 thru 0x3f57d64 (0x746d bytes) 2024-04-24T18:07:58.348 INFO:teuthology.orchestra.run.smithi049.stdout:1373 writesame 0x897edaa thru 0x89828e9 data_size 0x4f(0x3b40 bytes) 2024-04-24T18:07:58.348 INFO:teuthology.orchestra.run.smithi049.stdout:1374 punch from 0x72e9537 to 0x72eb85e, (0x2327 bytes) 2024-04-24T18:07:58.349 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.349 INFO:teuthology.orchestra.run.smithi049.stdout:1376 read 0x79253a3 thru 0x792a9ca (0x5628 bytes) 2024-04-24T18:07:58.358 INFO:teuthology.orchestra.run.smithi049.stdout:1377 writesame 0xd133ca4 thru 0xd141ac8 data_size 0x1d(0xde25 bytes) 2024-04-24T18:07:58.359 INFO:teuthology.orchestra.run.smithi049.stdout:1378 read 0x975234a thru 0x975d053 (0xad0a bytes) 2024-04-24T18:07:58.359 INFO:teuthology.orchestra.run.smithi049.stdout:1379 read 0xb79eac4 thru 0xb7a6071 (0x75ae bytes) 2024-04-24T18:07:58.359 INFO:teuthology.orchestra.run.smithi049.stdout:1381 punch from 0xa2280e6 to 0xa23386c, (0xb786 bytes) 2024-04-24T18:07:58.359 INFO:teuthology.orchestra.run.smithi049.stdout:1384 writesame 0xca3fb3d thru 0xca4eb94 data_size 0x4(0xf058 bytes) 2024-04-24T18:07:58.360 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:07:58.367+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:07:58.365 INFO:teuthology.orchestra.run.smithi049.stdout:1385 read 0xfddd79 thru 0xfe812a (0xa3b2 bytes) 2024-04-24T18:07:58.366 INFO:teuthology.orchestra.run.smithi049.stdout:1388 write 0x6d4df3c thru 0x6d5a592 (0xc657 bytes) 2024-04-24T18:07:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:1391 write 0xdc99ace thru 0xdca0976 (0x6ea9 bytes) 2024-04-24T18:07:58.380 INFO:teuthology.orchestra.run.smithi049.stdout:1392 read 0x9743658 thru 0x97480d0 (0x4a79 bytes) 2024-04-24T18:07:58.380 INFO:teuthology.orchestra.run.smithi049.stdout:1393 write 0xd1bbd7e thru 0xd1c6a70 (0xacf3 bytes) 2024-04-24T18:07:58.384 INFO:teuthology.orchestra.run.smithi049.stdout:1394 read 0x8e3c5f4 thru 0x8e450e3 (0x8af0 bytes) 2024-04-24T18:07:58.384 INFO:teuthology.orchestra.run.smithi049.stdout:1395 write 0xafab79f thru 0xafb1ee5 (0x6747 bytes) 2024-04-24T18:07:58.387 INFO:teuthology.orchestra.run.smithi049.stdout:1396 writesame 0x77ce47 thru 0x77ff6e data_size 0x34(0x3128 bytes) 2024-04-24T18:07:58.394 INFO:teuthology.orchestra.run.smithi049.stdout:1397 write 0x7540fc5 thru 0x754c930 (0xb96c bytes) 2024-04-24T18:07:58.397 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.397 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.410 INFO:teuthology.orchestra.run.smithi049.stdout:1401 trunc from 0xdca0977 to 0x2b74272 2024-04-24T18:07:58.421 INFO:teuthology.orchestra.run.smithi049.stdout:1403 read 0x41c352 thru 0x4214bb (0x516a bytes) 2024-04-24T18:07:58.422 INFO:teuthology.orchestra.run.smithi049.stdout:1405 read 0x1dd319e thru 0x1de16be (0xe521 bytes) 2024-04-24T18:07:58.422 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.422 INFO:teuthology.orchestra.run.smithi049.stdout:1407 read 0x282dae0 thru 0x2831b65 (0x4086 bytes) 2024-04-24T18:07:58.422 INFO:teuthology.orchestra.run.smithi049.stdout:1408 punch from 0x1b475fd to 0x1b543cf, (0xcdd2 bytes) 2024-04-24T18:07:58.426 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:07:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:07:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:07:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40940 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72478564099999998, 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036446457000000002, 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40940 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707548+0000", 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.251e-06 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707551+0000", 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0019999999999998e-06 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707554+0000", 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743860+0000", 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036306756000000003 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743915+0000", 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4423999999999999e-05 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743994+0000", 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9207000000000002e-05 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40941 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1490944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72475208999999996, 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036634427999999997, 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40941 2024-04-24T18:07:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707583+0000", 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2089999999999999e-06 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707587+0000", 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1670000000000001e-06 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707590+0000", 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.554e-06 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744033+0000", 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036443542000000002 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744112+0000", 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8578999999999994e-05 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744215+0000", 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103607 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40942 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1523712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72474104399999995, 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036756906999999998, 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40942 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707594+0000", 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7090000000000001e-06 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707596+0000", 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707598+0000", 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9149999999999999e-06 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744245+0000", 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036646570000000003 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744295+0000", 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0343000000000002e-05 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744349+0000", 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3661e-05 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40943 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72473236500000005, 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036956462000000002, 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40943 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:07:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707600+0000", 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707602+0000", 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.215e-06 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707605+0000", 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0429999999999999e-06 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707607+0000", 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-06 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744380+0000", 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036773477999999998 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744464+0000", 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4048000000000006e-05 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744557+0000", 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3043999999999999e-05 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40944 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1589248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72472436500000004, 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037054089999999998, 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40944 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707610+0000", 2024-04-24T18:07:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1319999999999999e-06 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707613+0000", 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707614+0000", 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8819999999999999e-06 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744569+0000", 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036954539000000002 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744613+0000", 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3671999999999998e-05 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744663+0000", 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0145000000000002e-05 2024-04-24T18:07:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40945 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72471646499999998, 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037248315999999997, 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40945 2024-04-24T18:07:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707616+0000", 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707620+0000", 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1999999999999999e-06 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707623+0000", 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1669999999999999e-06 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707625+0000", 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1110000000000002e-06 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744693+0000", 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037068564999999998 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744773+0000", 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9172999999999996e-05 2024-04-24T18:07:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744865+0000", 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2305000000000005e-05 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40946 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72470592199999995, 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037340299, 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40946 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707628+0000", 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2699999999999999e-06 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707631+0000", 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9519999999999999e-06 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707633+0000", 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8619999999999999e-06 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744876+0000", 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037242550999999999 2024-04-24T18:07:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744918+0000", 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2546000000000002e-05 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744967+0000", 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9320999999999998e-05 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40947 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72469786599999997, 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037537860999999999, 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40947 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:07:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707636+0000", 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.186e-06 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707639+0000", 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707641+0000", 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.86e-06 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744997+0000", 2024-04-24T18:07:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037356364000000003 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745076+0000", 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8355000000000005e-05 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745173+0000", 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7379000000000004e-05 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40948 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1720320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72468994200000003, 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037655182000000002, 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40948 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707644+0000", 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.269e-06 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707647+0000", 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9359999999999999e-06 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707649+0000", 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.815e-06 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745200+0000", 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037551189999999998 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745246+0000", 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6214999999999997e-05 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745298+0000", 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1960999999999999e-05 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40949 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72468191299999996, 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037846454000000002, 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40949 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707652+0000", 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.13e-06 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707655+0000", 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707658+0000", 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5770000000000001e-06 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745329+0000", 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037671351999999998 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745408+0000", 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745498+0000", 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0043000000000005e-05 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40950 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72467323299999997, 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037946947000000002, 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40950 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707661+0000", 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.15e-06 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707664+0000", 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.965e-06 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707666+0000", 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745508+0000", 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037841977999999998 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745555+0000", 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7098999999999997e-05 2024-04-24T18:07:58.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745607+0000", 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1591999999999997e-05 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40951 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72466475299999999, 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038137931999999999, 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.473 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40951 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707670+0000", 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7010000000000001e-06 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707673+0000", 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9119999999999998e-06 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707675+0000", 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9429999999999999e-06 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745637+0000", 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037962379999999997 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745715+0000", 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7488000000000001e-05 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745806+0000", 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1723000000000003e-05 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40952 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72465612700000004, 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038236762000000001, 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40952 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707678+0000", 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.093e-06 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707681+0000", 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707683+0000", 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.951e-06 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745817+0000", 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038133854000000002 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745862+0000", 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5027000000000001e-05 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745914+0000", 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2120999999999998e-05 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40953 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72464809399999996, 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038424458000000002, 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.481 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40953 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707686+0000", 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.567e-06 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707689+0000", 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9220000000000001e-06 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707691+0000", 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9369999999999998e-06 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745943+0000", 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038252047999999997 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746018+0000", 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4938000000000007e-05 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746109+0000", 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1255000000000006e-05 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40954 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1916928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72463958500000003, 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038528139000000003, 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.485 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40954 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:07:58.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707693+0000", 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.065e-06 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707697+0000", 2024-04-24T18:07:58.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.881e-06 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707699+0000", 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.88e-06 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746120+0000", 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038420937000000002 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746171+0000", 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0555e-05 2024-04-24T18:07:58.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746222+0000", 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1025e-05 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40955 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72463176500000004, 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718855000000003, 2024-04-24T18:07:58.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40955 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707702+0000", 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.212e-06 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707706+0000", 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.534e-06 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707708+0000", 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746254+0000", 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038545870000000003 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746331+0000", 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7805000000000005e-05 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746420+0000", 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8721000000000007e-05 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40956 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72462314699999997, 2024-04-24T18:07:58.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038823347000000001, 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40956 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707711+0000", 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1659999999999999e-06 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707714+0000", 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.937e-06 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707716+0000", 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9920000000000002e-06 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746435+0000", 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718869000000003 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746482+0000", 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6882999999999998e-05 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746533+0000", 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1711e-05 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40957 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.497 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72461487300000005, 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039015052000000001, 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40957 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707721+0000", 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3110000000000001e-06 2024-04-24T18:07:58.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707725+0000", 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2669999999999998e-06 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707727+0000", 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746563+0000", 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038836751000000003 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746641+0000", 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7900999999999998e-05 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746733+0000", 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2015999999999994e-05 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40958 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2048000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.501 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72460418000000004, 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039110399999999997, 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40958 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707730+0000", 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2640000000000001e-06 2024-04-24T18:07:58.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707733+0000", 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9170000000000002e-06 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707735+0000", 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746744+0000", 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039008958000000003 2024-04-24T18:07:58.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746789+0000", 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5019999999999999e-05 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746839+0000", 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0531999999999997e-05 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40959 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:07:58.505 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.72459600700000004, 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039290798000000002, 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40959 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707738+0000", 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.24e-06 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707742+0000", 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6200000000000001e-06 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707744+0000", 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.615e-06 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746859+0000", 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039115095000000003 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746938+0000", 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.747028+0000", 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9742999999999998e-05 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:07:58.509 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:58.510 INFO:teuthology.orchestra.run.smithi049.stdout:1409 write 0xe44f767 thru 0xe45adbf (0xb659 bytes) 2024-04-24T18:07:58.511 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:58.519+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:58.512 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1410 read 0x5d56fb7 thru 0x5d5c918 (0x5962 bytes) 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1411 writesame 0x6f7fb9 thru 0x6f8dbf data_size 0x9(0xe07 bytes) 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1413 write 0x630eca thru 0x63a2dd (0x9414 bytes) 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1415 read 0x152f6f6 thru 0x152fe8f (0x79a bytes) 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1417 trunc from 0xe45adc0 to 0x4b31044 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1418 read 0xc83e15 thru 0xc91f3f (0xe12b bytes) 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1419 write 0xa83e04a thru 0xa83ee6b (0xe22 bytes) 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1420 write 0x7f04c2b thru 0x7f0ce75 (0x824b bytes) 2024-04-24T18:07:58.513 INFO:teuthology.orchestra.run.smithi049.stdout:1421 write 0xe2af8f9 thru 0xe2bbaff (0xc207 bytes) 2024-04-24T18:07:58.514 INFO:teuthology.orchestra.run.smithi049.stdout:1422 write 0xd59b4eb thru 0xd5a6ac7 (0xb5dd bytes) 2024-04-24T18:07:58.518 INFO:teuthology.orchestra.run.smithi049.stdout:1424 read 0x6fdd068 thru 0x6fdf6ca (0x2663 bytes) 2024-04-24T18:07:58.526 INFO:teuthology.orchestra.run.smithi049.stdout:1425 trunc from 0xe2bbb00 to 0x6b3ee7c 2024-04-24T18:07:58.537 INFO:teuthology.orchestra.run.smithi049.stdout:1426 write 0x68c3cea thru 0x68cafad (0x72c4 bytes) 2024-04-24T18:07:58.541 INFO:teuthology.orchestra.run.smithi049.stdout:1427 write 0x1fa72f0 thru 0x1fb173a (0xa44b bytes) 2024-04-24T18:07:58.545 INFO:teuthology.orchestra.run.smithi049.stdout:1428 trunc from 0x6b3ee7c to 0x621065c 2024-04-24T18:07:58.554 INFO:teuthology.orchestra.run.smithi049.stdout:1429 read 0x3ce6ad9 thru 0x3cf0a05 (0x9f2d bytes) 2024-04-24T18:07:58.554 INFO:teuthology.orchestra.run.smithi049.stdout:1430 read 0x2fe8d75 thru 0x2fef3d7 (0x6663 bytes) 2024-04-24T18:07:58.561 INFO:teuthology.orchestra.run.smithi049.stdout:1431 trunc from 0x621065c to 0x63ffca 2024-04-24T18:07:58.610 INFO:teuthology.orchestra.run.smithi049.stdout:1433 trunc from 0x63ffca to 0xc97b676 2024-04-24T18:07:58.611 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:58.618+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:58.614 INFO:teuthology.orchestra.run.smithi049.stdout:1434 write 0x8bb894 thru 0x8c0627 (0x4d94 bytes) 2024-04-24T18:07:58.622 INFO:teuthology.orchestra.run.smithi049.stdout:1435 read 0x48d2278 thru 0x48da24a (0x7fd3 bytes) 2024-04-24T18:07:58.622 INFO:teuthology.orchestra.run.smithi049.stdout:1436 read 0x433652b thru 0x433afbf (0x4a95 bytes) 2024-04-24T18:07:58.622 INFO:teuthology.orchestra.run.smithi049.stdout:1437 read 0xaf493cd thru 0xaf51d67 (0x899b bytes) 2024-04-24T18:07:58.622 INFO:teuthology.orchestra.run.smithi049.stdout:1438 read 0x7c115dc thru 0x7c1aa68 (0x948d bytes) 2024-04-24T18:07:58.622 INFO:teuthology.orchestra.run.smithi049.stdout:1439 writesame 0x5f9bdb1 thru 0x5fa97d9 data_size 0x1(0xda29 bytes) 2024-04-24T18:07:58.670 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:58.670 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:58.670 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:07:58.671 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:58.681 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:07:58.722 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:58.730+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:58.778 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:07:58.822 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:58.830+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:58.851 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:07:58.852 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:07:58.852 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:07:58.852 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:07:58.852 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:07:58.862 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:07:58.923 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:07:58.923 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:07:58.931+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:07:58.942 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:07:58.960 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:07:58.979 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:07:58.998 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:07:59.015 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:59.023+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:59.016 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:07:59.032 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:59.033 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:59.035 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:07:59.044 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:07:59.054 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:07:59.056 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:07:59.116 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:59.124+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:59.183 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:59.183 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:59.193 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:07:59.217 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:07:59.226+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:07:59.319 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:59.326+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:07:59.362 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:59.362 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:59.372 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:07:59.420 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:07:59.428+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:07:59.520 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:59.529+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:59.541 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:59.541 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:59.551 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:07:59.621 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:07:59.629+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:07:59.634 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:07:59.651 INFO:teuthology.orchestra.run.smithi049.stdout:1441 write 0xd8e00e9 thru 0xd8e97e3 (0x96fb bytes) 2024-04-24T18:07:59.654 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:59.661 INFO:teuthology.orchestra.run.smithi049.stdout:1443 trunc from 0xd8e97e4 to 0x8015c0a 2024-04-24T18:07:59.682 INFO:teuthology.orchestra.run.smithi049.stdout:1445 writesame 0xd555233 thru 0xd560696 data_size 0xa(0xb464 bytes) 2024-04-24T18:07:59.682 INFO:teuthology.orchestra.run.smithi049.stdout:1446 write 0x35d0903 thru 0x35d65b1 (0x5caf bytes) 2024-04-24T18:07:59.686 INFO:teuthology.orchestra.run.smithi049.stdout:1448 read 0x8efcb26 thru 0x8efccb7 (0x192 bytes) 2024-04-24T18:07:59.686 INFO:teuthology.orchestra.run.smithi049.stdout:1451 punch from 0xc131228 to 0xc131b61, (0x939 bytes) 2024-04-24T18:07:59.686 INFO:teuthology.orchestra.run.smithi049.stdout:1452 writesame 0x6c227ce thru 0x6c22f7f data_size 0x5(0x7b2 bytes) 2024-04-24T18:07:59.690 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:59.690 INFO:teuthology.orchestra.run.smithi049.stdout:1454 read 0x1352ead thru 0x135bee5 (0x9039 bytes) 2024-04-24T18:07:59.690 INFO:teuthology.orchestra.run.smithi049.stdout:1457 write 0x3847c90 thru 0x385620a (0xe57b bytes) 2024-04-24T18:07:59.693 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:59.693 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:59.699 INFO:teuthology.orchestra.run.smithi049.stdout:1458 trunc from 0xd560697 to 0x827bb6e 2024-04-24T18:07:59.703 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:07:59.704 INFO:teuthology.orchestra.run.smithi049.stdout:1459 writesame 0x2a60199 thru 0x2a66e7e data_size 0x2(0x6ce6 bytes) 2024-04-24T18:07:59.713 INFO:teuthology.orchestra.run.smithi049.stdout:1460 writesame 0x4d067ff thru 0x4d07583 data_size 0x1(0xd85 bytes) 2024-04-24T18:07:59.722 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:07:59.730+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:07:59.728 INFO:teuthology.orchestra.run.smithi049.stdout:1461 write 0xcbe1896 thru 0xcbec323 (0xaa8e bytes) 2024-04-24T18:07:59.730 INFO:teuthology.orchestra.run.smithi049.stdout:1462 read 0x18789d5 thru 0x1880d8c (0x83b8 bytes) 2024-04-24T18:07:59.731 INFO:teuthology.orchestra.run.smithi049.stdout:1463 write 0x10f6827 thru 0x1105fc4 (0xf79e bytes) 2024-04-24T18:07:59.735 INFO:teuthology.orchestra.run.smithi049.stdout:1466 write 0x9ff402f thru 0xa0038c9 (0xf89b bytes) 2024-04-24T18:07:59.739 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:07:59.748 INFO:teuthology.orchestra.run.smithi049.stdout:1468 trunc from 0xcbec324 to 0x4b8affe 2024-04-24T18:07:59.761 INFO:teuthology.orchestra.run.smithi049.stdout:1469 punch from 0x1a6a67f to 0x1a71246, (0x6bc7 bytes) 2024-04-24T18:07:59.761 INFO:teuthology.orchestra.run.smithi049.stdout:1471 write 0xf56505 thru 0xf5b1fb (0x4cf7 bytes) 2024-04-24T18:07:59.776 INFO:teuthology.orchestra.run.smithi049.stdout:1472 write 0xa933236 thru 0xa93881e (0x55e9 bytes) 2024-04-24T18:07:59.779 INFO:teuthology.orchestra.run.smithi049.stdout:1473 read 0x72964fc thru 0x7298837 (0x233c bytes) 2024-04-24T18:07:59.779 INFO:teuthology.orchestra.run.smithi049.stdout:1474 writesame 0x3232322 thru 0x3240329 data_size 0x86(0xe008 bytes) 2024-04-24T18:07:59.783 INFO:teuthology.orchestra.run.smithi049.stdout:1477 clone 11 order 25 su 262144 sc 4 2024-04-24T18:07:59.823 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:07:59.831+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:07:59.870 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:07:59.870 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:07:59.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:07:59.923 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:07:59.931+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:00.024 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:00.032+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:00.044 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:00.044 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:00.054 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:00.125 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:00.133+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:00.177 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:00.177 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:00.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:00.226 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:00.234+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:00.310 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:00.310 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:00.320 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:00.364 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:00.372+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:00.428 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:00.436+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:00.444 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:00.444 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:00.454 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:00.537 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:00.546+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:00.618 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:00.619 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:00.629 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:08:00.630 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:00.638+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:00.630 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:08:00.630 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:08:00.630 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-04-24T18:08:00.630 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-24T18:08:00.631 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:08:00.696 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone10 from 0xa93881f (overlap 0x63ffca) to 0x13fff5 2024-04-24T18:08:00.731 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:00.739+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:00.770 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #9, image image_client.0-clone9 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent10 2024-04-24T18:08:00.787 INFO:teuthology.orchestra.run.smithi049.stdout:1478 write 0xd064b73 thru 0xd072e1a (0xe2a8 bytes) 2024-04-24T18:08:00.790 INFO:teuthology.orchestra.run.smithi049.stdout:1479 read 0x297b154 thru 0x2983be0 (0x8a8d bytes) 2024-04-24T18:08:00.791 INFO:teuthology.orchestra.run.smithi049.stdout:1480 write 0xba9815 thru 0xbb4547 (0xad33 bytes) 2024-04-24T18:08:00.804 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:00.804 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:00.811 INFO:teuthology.orchestra.run.smithi049.stdout:1481 trunc from 0xd072e1b to 0x47f8fd3 2024-04-24T18:08:00.815 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:08:00.832 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:00.839+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:00.839 INFO:teuthology.orchestra.run.smithi049.stdout:1483 read 0x326513f thru 0x326d9d3 (0x8895 bytes) 2024-04-24T18:08:00.839 INFO:teuthology.orchestra.run.smithi049.stdout:1484 punch from 0x3925e1 to 0x3a223d, (0xfc5c bytes) 2024-04-24T18:08:00.839 INFO:teuthology.orchestra.run.smithi049.stdout:1486 writesame 0x2e17f22 thru 0x2e27673 data_size 0x1(0xf752 bytes) 2024-04-24T18:08:00.895 INFO:teuthology.orchestra.run.smithi049.stdout:1487 write 0xee6a06a thru 0xee6b27f (0x1216 bytes) 2024-04-24T18:08:00.898 INFO:teuthology.orchestra.run.smithi049.stdout:1488 read 0xa3094f2 thru 0xa315466 (0xbf75 bytes) 2024-04-24T18:08:00.898 INFO:teuthology.orchestra.run.smithi049.stdout:1489 write 0xc9fe9cc thru 0xca08820 (0x9e55 bytes) 2024-04-24T18:08:00.902 INFO:teuthology.orchestra.run.smithi049.stdout:1490 read 0x16b5a6a thru 0x16b9588 (0x3b1f bytes) 2024-04-24T18:08:00.902 INFO:teuthology.orchestra.run.smithi049.stdout:1492 read 0xa7335aa thru 0xa740608 (0xd05f bytes) 2024-04-24T18:08:00.902 INFO:teuthology.orchestra.run.smithi049.stdout:1493 writesame 0x597769e thru 0x59863d7 data_size 0x5(0xed3a bytes) 2024-04-24T18:08:00.906 INFO:teuthology.orchestra.run.smithi049.stdout:1494 punch from 0xa1e73b9 to 0xa1f1e73, (0xaaba bytes) 2024-04-24T18:08:00.906 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:00.906 INFO:teuthology.orchestra.run.smithi049.stdout:1498 punch from 0xd3bb65e to 0xd3c2650, (0x6ff2 bytes) 2024-04-24T18:08:00.906 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:00.907 INFO:teuthology.orchestra.run.smithi049.stdout:1500 write 0x9aa75b5 thru 0x9ab5917 (0xe363 bytes) 2024-04-24T18:08:00.909 INFO:teuthology.orchestra.run.smithi049.stdout:1502 read 0x95b605e thru 0x95c4bd2 (0xeb75 bytes) 2024-04-24T18:08:00.910 INFO:teuthology.orchestra.run.smithi049.stdout:1503 punch from 0xaabeaa1 to 0xaacd5e6, (0xeb45 bytes) 2024-04-24T18:08:00.910 INFO:teuthology.orchestra.run.smithi049.stdout:1504 read 0xeb063c4 thru 0xeb149a2 (0xe5df bytes) 2024-04-24T18:08:00.910 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:00.918 INFO:teuthology.orchestra.run.smithi049.stdout:1508 trunc from 0xee6b280 to 0x7ce8c15 2024-04-24T18:08:00.929 INFO:teuthology.orchestra.run.smithi049.stdout:1509 write 0x5f05e54 thru 0x5f0bebc (0x6069 bytes) 2024-04-24T18:08:00.930 INFO:teuthology.orchestra.run.smithi049.stdout:1510 trunc from 0x7ce8c15 to 0x7f1f684 2024-04-24T18:08:00.932 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:00.940+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:00.941 INFO:teuthology.orchestra.run.smithi049.stdout:1512 trunc from 0x7f1f684 to 0x275ecf2 2024-04-24T18:08:00.966 INFO:teuthology.orchestra.run.smithi049.stdout:1513 write 0xd6e3577 thru 0xd6f2644 (0xf0ce bytes) 2024-04-24T18:08:00.969 INFO:teuthology.orchestra.run.smithi049.stdout:1514 read 0x86246af thru 0x862a7a9 (0x60fb bytes) 2024-04-24T18:08:00.969 INFO:teuthology.orchestra.run.smithi049.stdout:1515 write 0x6102ce4 thru 0x6102f7b (0x298 bytes) 2024-04-24T18:08:00.970 INFO:teuthology.orchestra.run.smithi049.stdout:1516 punch from 0x2b621c2 to 0x2b64765, (0x25a3 bytes) 2024-04-24T18:08:00.970 INFO:teuthology.orchestra.run.smithi049.stdout:1517 read 0x787c169 thru 0x787d509 (0x13a1 bytes) 2024-04-24T18:08:00.970 INFO:teuthology.orchestra.run.smithi049.stdout:1519 write 0xa14591a thru 0xa150ae8 (0xb1cf bytes) 2024-04-24T18:08:00.975 INFO:teuthology.orchestra.run.smithi049.stdout:1521 trunc from 0xd6f2645 to 0xb88567c 2024-04-24T18:08:00.982 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:00.982 INFO:teuthology.orchestra.run.smithi049.stdout:1524 read 0x4816844 thru 0x482603e (0xf7fb bytes) 2024-04-24T18:08:00.984 INFO:teuthology.orchestra.run.smithi049.stdout:1527 trunc from 0xb88567c to 0x9c5d229 2024-04-24T18:08:00.987 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:00.987 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:00.991 INFO:teuthology.orchestra.run.smithi049.stdout:1529 read 0x3d21f92 thru 0x3d2d40c (0xb47b bytes) 2024-04-24T18:08:00.998 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:01.002 INFO:teuthology.orchestra.run.smithi049.stdout:1530 write 0xe13a963 thru 0xe146b16 (0xc1b4 bytes) 2024-04-24T18:08:01.004 INFO:teuthology.orchestra.run.smithi049.stdout:1531 write 0x4f71eba thru 0x4f7c792 (0xa8d9 bytes) 2024-04-24T18:08:01.006 INFO:teuthology.orchestra.run.smithi049.stdout:1532 read 0x7527dc2 thru 0x752d8e0 (0x5b1f bytes) 2024-04-24T18:08:01.017 INFO:teuthology.orchestra.run.smithi049.stdout:1534 trunc from 0xe146b17 to 0x4b64029 2024-04-24T18:08:01.029 INFO:teuthology.orchestra.run.smithi049.stdout:1536 trunc from 0x4b64029 to 0x5f64d34 2024-04-24T18:08:01.033 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:01.041+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:01.035 INFO:teuthology.orchestra.run.smithi049.stdout:1538 trunc from 0x5f64d34 to 0x7b68a35 2024-04-24T18:08:01.037 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:08:01.037 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 50, 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "fsid": "5dae32d6-9101-4f7b-a6ef-ca5186fee71b", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "created": "2024-04-24T18:07:07.515859+0000", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "modified": "2024-04-24T18:08:00.628716+0000", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "last_up_change": "2024-04-24T18:07:17.349468+0000", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "last_in_change": "2024-04-24T18:07:55.610486+0000", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_num": 5801984, 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_set": [ 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "noscrub", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "pglog_hardlimit", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "purged_snapdirs", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_deletes", 2024-04-24T18:08:01.038 INFO:teuthology.orchestra.run.smithi049.stdout: "sortbitwise" 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_version": 4, 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "full_ratio": 0.94999998807907104, 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_snapshot": "", 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_max": 4, 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd": 8, 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "require_min_compat_client": "luminous", 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "min_compat_client": "jewel", 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "require_osd_release": "quincy", 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "pools": [ 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 1, 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "rbd", 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:18.850062+0000", 2024-04-24T18:08:01.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 8, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 8, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 8, 2024-04-24T18:08:01.040 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 8, 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 8, 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "21", 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "21", 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:08:01.041 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 19, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [ 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: 3 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": 3, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": 3, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:08:01.042 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:08:01.043 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 2, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": ".mgr", 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:19.353026+0000", 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:08:01.044 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 1, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 1, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 1, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 1, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 1, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:08:01.045 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "17", 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:08:01.046 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:08:01.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_max": 32, 2024-04-24T18:08:01.048 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 1 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": {} 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 3, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "cache", 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:25.800822+0000", 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 9, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool,incomplete_clones", 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:08:01.049 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 4, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 4, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 4, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 4, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 4, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:08:01.050 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "25", 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "21", 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 19, 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:08:01.051 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": 1, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "writeback", 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 250, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bloom", 2024-04-24T18:08:01.052 INFO:teuthology.orchestra.run.smithi049.stdout: "false_positive_probability": 0.050000000000000003, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "target_size": 0, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "seed": 0 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 60, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 8, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [ 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:08:01.053 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 4, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "pool_client.0", 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:35.513947+0000", 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 8193, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:08:01.055 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 32, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 32, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 32, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 32, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 32, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "50", 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-04-24T18:08:01.056 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 14, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 50, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:08:01.057 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:08:01.058 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osds": [ 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "24da2d2a-953e-4440-9f6d-fb19cbec9ca1", 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.059 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 48, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.060 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6803", 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6805", 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6807", 2024-04-24T18:08:01.061 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6813", 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6815", 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.062 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6809", 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6811", 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6803/43474", 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6807/43474", 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6815/43474", 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6811/43474", 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "a47a3673-abbd-4827-966a-d68b17f3a18f", 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.064 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 45, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6802", 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6804", 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.066 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6806", 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6812", 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6814", 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6808", 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6810", 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6802/43473", 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6806/43473", 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6814/43473", 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6810/43473", 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "de3910ae-cc81-45d4-8639-cf2415330e64", 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.069 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 45, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6824", 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.070 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6825", 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6826", 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6827", 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6830", 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6831", 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6828", 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6829", 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.073 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6825/43481", 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6827/43481", 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6831/43481", 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6829/43481", 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0", 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.074 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 45, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6816", 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6817", 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6818", 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6819", 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6822", 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6823", 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.077 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6820", 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6821", 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6817/43475", 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6819/43475", 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6823/43475", 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6821/43475", 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "47f1fd84-e601-4e48-8058-bfa9296a9f90", 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.079 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 45, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6800", 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6801", 2024-04-24T18:08:01.080 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6802", 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6803", 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.081 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6806", 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6807", 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.082 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6804", 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6805", 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.083 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6801/43499", 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6803/43499", 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6807/43499", 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6805/43499", 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "1796d83b-591c-4c9d-b376-e36a2416a17e", 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.084 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 27, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6808", 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6809", 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6810", 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6811", 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6814", 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.087 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6815", 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6812", 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6813", 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.088 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6809/43500", 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6811/43500", 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6815/43500", 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6813/43500", 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "5f203bac-3db4-4129-96cc-63dc06ee5ffa", 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.089 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 45, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.090 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6824", 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6825", 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6826", 2024-04-24T18:08:01.091 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6827", 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6830", 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6831", 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6828", 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6829", 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43544 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6825/43544", 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6827/43544", 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6831/43544", 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6829/43544", 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "51061805-fbe5-4e80-be6b-dfc37cec0221", 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 48, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6816", 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6817", 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6818", 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6819", 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.097 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6822", 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6823", 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:08:01.098 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6820", 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6821", 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6817/43501", 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6819/43501", 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6823/43501", 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6821/43501", 2024-04-24T18:08:01.099 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_xinfo": [ 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.248445+0000", 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.246633+0000", 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.101 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.176542+0000", 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.192111+0000", 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.151845+0000", 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.103 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.187089+0000", 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.255937+0000", 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.104 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.160381+0000", 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap": [], 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap_items": [], 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_temp": [], 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_temp": [], 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklist": {}, 2024-04-24T18:08:01.105 INFO:teuthology.orchestra.run.smithi049.stdout: "range_blocklist": {}, 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profiles": { 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "default": { 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-failure-domain": "osd", 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "k": "2", 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "m": "1", 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin": "jerasure", 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "technique": "reed_sol_van" 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps_queue": [], 2024-04-24T18:08:01.106 INFO:teuthology.orchestra.run.smithi049.stdout: "new_removed_snaps": [], 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "new_purged_snaps": [], 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_node_flags": {}, 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "device_class_flags": {}, 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode": { 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_enabled": false, 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_bucket_count": 0, 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "degraded_stretch_mode": 0, 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "recovering_stretch_mode": 0, 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_bucket": 0 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.107 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:01.108 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:01.108 INFO:teuthology.orchestra.run.smithi049.stdout:1540 write 0xba11754 thru 0xba1520f (0x3abc bytes) 2024-04-24T18:08:01.110 INFO:tasks.thrashosds.thrasher:j is {'epoch': 50, 'fsid': '5dae32d6-9101-4f7b-a6ef-ca5186fee71b', 'created': '2024-04-24T18:07:07.515859+0000', 'modified': '2024-04-24T18:08:00.628716+0000', 'last_up_change': '2024-04-24T18:07:17.349468+0000', 'last_in_change': '2024-04-24T18:07:55.610486+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-24T18:07:18.850062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-04-24T18:07:19.353026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-04-24T18:07:25.800822+0000', 'flags': 9, 'flags_names': 'hashpspool,incomplete_clones', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-04-24T18:07:35.513947+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '50', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 14, 'snap_epoch': 50, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '24da2d2a-953e-4440-9f6d-fb19cbec9ca1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6801', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6803', 'nonce': 43474}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6805', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6807', 'nonce': 43474}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6813', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6815', 'nonce': 43474}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6809', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6811', 'nonce': 43474}]}, 'public_addr': '172.21.15.49:6803/43474', 'cluster_addr': '172.21.15.49:6807/43474', 'heartbeat_back_addr': '172.21.15.49:6815/43474', 'heartbeat_front_addr': '172.21.15.49:6811/43474', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a47a3673-abbd-4827-966a-d68b17f3a18f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6800', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6802', 'nonce': 43473}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6804', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6806', 'nonce': 43473}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6812', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6814', 'nonce': 43473}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6808', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6810', 'nonce': 43473}]}, 'public_addr': '172.21.15.49:6802/43473', 'cluster_addr': '172.21.15.49:6806/43473', 'heartbeat_back_addr': '172.21.15.49:6814/43473', 'heartbeat_front_addr': '172.21.15.49:6810/43473', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'de3910ae-cc81-45d4-8639-cf2415330e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6824', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6825', 'nonce': 43481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6826', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6827', 'nonce': 43481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6830', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6831', 'nonce': 43481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6828', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6829', 'nonce': 43481}]}, 'public_addr': '172.21.15.49:6825/43481', 'cluster_addr': '172.21.15.49:6827/43481', 'heartbeat_back_addr': '172.21.15.49:6831/43481', 'heartbeat_front_addr': '172.21.15.49:6829/43481', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6816', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6817', 'nonce': 43475}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6818', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6819', 'nonce': 43475}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6822', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6823', 'nonce': 43475}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6820', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6821', 'nonce': 43475}]}, 'public_addr': '172.21.15.49:6817/43475', 'cluster_addr': '172.21.15.49:6819/43475', 'heartbeat_back_addr': '172.21.15.49:6823/43475', 'heartbeat_front_addr': '172.21.15.49:6821/43475', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '47f1fd84-e601-4e48-8058-bfa9296a9f90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6800', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6801', 'nonce': 43499}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6802', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6803', 'nonce': 43499}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6806', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6807', 'nonce': 43499}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6804', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6805', 'nonce': 43499}]}, 'public_addr': '172.21.15.78:6801/43499', 'cluster_addr': '172.21.15.78:6803/43499', 'heartbeat_back_addr': '172.21.15.78:6807/43499', 'heartbeat_front_addr': '172.21.15.78:6805/43499', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1796d83b-591c-4c9d-b376-e36a2416a17e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 27, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6808', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6809', 'nonce': 43500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6810', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6811', 'nonce': 43500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6814', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6815', 'nonce': 43500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6812', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6813', 'nonce': 43500}]}, 'public_addr': '172.21.15.78:6809/43500', 'cluster_addr': '172.21.15.78:6811/43500', 'heartbeat_back_addr': '172.21.15.78:6815/43500', 'heartbeat_front_addr': '172.21.15.78:6813/43500', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5f203bac-3db4-4129-96cc-63dc06ee5ffa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6824', 'nonce': 43544}, {'type': 'v1', 'addr': '172.21.15.78:6825', 'nonce': 43544}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6826', 'nonce': 43544}, {'type': 'v1', 'addr': '172.21.15.78:6827', 'nonce': 43544}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6830', 'nonce': 43544}, {'type': 'v1', 'addr': '172.21.15.78:6831', 'nonce': 43544}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6828', 'nonce': 43544}, {'type': 'v1', 'addr': '172.21.15.78:6829', 'nonce': 43544}]}, 'public_addr': '172.21.15.78:6825/43544', 'cluster_addr': '172.21.15.78:6827/43544', 'heartbeat_back_addr': '172.21.15.78:6831/43544', 'heartbeat_front_addr': '172.21.15.78:6829/43544', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '51061805-fbe5-4e80-be6b-dfc37cec0221', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6816', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6817', 'nonce': 43501}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6818', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6819', 'nonce': 43501}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6822', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6823', 'nonce': 43501}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6820', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6821', 'nonce': 43501}]}, 'public_addr': '172.21.15.78:6817/43501', 'cluster_addr': '172.21.15.78:6819/43501', 'heartbeat_back_addr': '172.21.15.78:6823/43501', 'heartbeat_front_addr': '172.21.15.78:6821/43501', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.248445+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.246633+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.176542+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.192111+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.151845+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.187089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.255937+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.160381+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-24T18:08:01.110 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-04-24T18:08:01.111 INFO:teuthology.orchestra.run.smithi049.stdout:1541 trunc from 0xba15210 to 0x5db3248 2024-04-24T18:08:01.111 INFO:teuthology.orchestra.run.smithi049.stdout:1543 writesame 0x21a2a43 thru 0x21a6a50 data_size 0x9(0x400e bytes) 2024-04-24T18:08:01.111 INFO:teuthology.orchestra.run.smithi049.stdout:1544 punch from 0x12bcad7 to 0x12c6ca7, (0xa1d0 bytes) 2024-04-24T18:08:01.112 INFO:teuthology.orchestra.run.smithi049.stdout:1545 read 0x1664e2b thru 0x1669d76 (0x4f4c bytes) 2024-04-24T18:08:01.112 INFO:teuthology.orchestra.run.smithi049.stdout:1546 punch from 0x4035f23 to 0x403bf51, (0x602e bytes) 2024-04-24T18:08:01.112 INFO:teuthology.orchestra.run.smithi049.stdout:1548 write 0xbb692be thru 0xbb6b542 (0x2285 bytes) 2024-04-24T18:08:01.112 INFO:teuthology.orchestra.run.smithi049.stdout:1549 read 0x67bbbb1 thru 0x67c3460 (0x78b0 bytes) 2024-04-24T18:08:01.112 INFO:teuthology.orchestra.run.smithi049.stdout:1551 writesame 0x19da166 thru 0x19dc644 data_size 0x1(0x24df bytes) 2024-04-24T18:08:01.112 INFO:teuthology.orchestra.run.smithi049.stdout:1552 write 0x815c905 thru 0x815e315 (0x1a11 bytes) 2024-04-24T18:08:01.112 INFO:teuthology.orchestra.run.smithi049.stdout:1553 write 0xcad43c3 thru 0xcade6a8 (0xa2e6 bytes) 2024-04-24T18:08:01.115 INFO:teuthology.orchestra.run.smithi049.stdout:1556 trunc from 0xcade6a9 to 0x3c733ed 2024-04-24T18:08:01.126 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:01.126 INFO:teuthology.orchestra.run.smithi049.stdout:1558 punch from 0x2ccc06e to 0x2cd7cde, (0xbc70 bytes) 2024-04-24T18:08:01.131 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:01.131 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:01.133 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:01.141+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:01.141 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:01.161 INFO:teuthology.orchestra.run.smithi049.stdout:1559 trunc from 0x3c733ed to 0xd577725 2024-04-24T18:08:01.166 INFO:teuthology.orchestra.run.smithi049.stdout:1560 write 0x5a43427 thru 0x5a48bc0 (0x579a bytes) 2024-04-24T18:08:01.167 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:01.178 INFO:teuthology.orchestra.run.smithi049.stdout:1563 trunc from 0xd577725 to 0x35b9584 2024-04-24T18:08:01.194 INFO:teuthology.orchestra.run.smithi049.stdout:1566 write 0x60a5dd9 thru 0x60aa160 (0x4388 bytes) 2024-04-24T18:08:01.195 INFO:teuthology.orchestra.run.smithi049.stdout:1567 clone 12 order 21 su 524288 sc 15 2024-04-24T18:08:01.234 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:01.242+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:01.269 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:01.287 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:01.306 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:01.307 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:01.307 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:01.317 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:01.325 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:01.335 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:01.343+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:01.344 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:01.362 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:01.381 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:01.400 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:01.401 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:01.402 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:01.402 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:01.402 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:01.402 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:01.402 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:01.444 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.835045721, 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:08:01.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:08:01.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:08:01.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:01.451 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834976669, 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:08:01.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834952254000001, 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:08:01.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:08:01.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834929105000001, 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:08:01.463 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:08:01.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:08:01.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:08:01.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834848536999999, 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.469 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:01.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:01.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:08:01.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:08:01.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:08:01.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:08:01.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:08:01.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:08:01.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:08:01.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:08:01.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:08:01.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:08:01.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:08:01.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:08:01.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:08:01.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:08:01.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:08:01.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:08:01.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:08:01.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:08:01.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:08:01.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:08:01.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:08:01.485 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.485 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.485 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.486 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.486 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:01.487 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834823727, 2024-04-24T18:08:01.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:08:01.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:08:01.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.488 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:01.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:08:01.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:08:01.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:08:01.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:08:01.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:08:01.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.495 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.496 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.496 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.496 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:01.496 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834803614999998, 2024-04-24T18:08:01.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:08:01.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:01.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:08:01.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834732952000003, 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:08:01.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:08:01.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:08:01.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:08:01.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:08:01.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.834721178999999, 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:08:01.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:08:01.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:08:01.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.414399349999997, 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:08:01.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:08:01.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:08:01.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:08:01.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:08:01.523 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.523 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.414352168000001, 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:01.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.414327047, 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.530 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:08:01.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.535 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.414274913, 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:08:01.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:08:01.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:08:01.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 40.414211449, 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:08:01.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:08:01.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 39.90304115, 2024-04-24T18:08:01.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073643390000000001, 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 156 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548239+0000", 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8899999999999996e-06 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548246+0000", 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9370000000000004e-06 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548259+0000", 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549586+0000", 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001326743 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549641+0000", 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5856999999999997e-05 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549713+0000", 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1970000000000004e-05 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550899+0000", 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001185567 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555532+0000", 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0046325760000000002 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555573+0000", 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1730000000000002e-05 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555595+0000", 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1324000000000001e-05 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 39.903005776999997, 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:08:01.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 39.902980749999998, 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.557 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:01.562 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 39.902901513000003, 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.567 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.6407127320000008, 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.571 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.6396066630000004, 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:08:01.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.575 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:01.576 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:01.576 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:01.577 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:01.461+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:01.578 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:01.586+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:01.578 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:01.677 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:01.686+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:01.696 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone11 from 0x60aa161 (overlap 0x275ecf2) to 0x3efe18 2024-04-24T18:08:01.720 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:01.720 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:01.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.720 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:44539 4.1e 4:7b616eff:::rbd_header.11747f0b518c:head [call rbd.get_data_pool in=16b out=8b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e51)", 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:01.728842+0000", 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00013375099999999999, 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014585999999999999, 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 44539 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728842+0000", 2024-04-24T18:08:01.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728842+0000", 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728841+0000", 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728845+0000", 2024-04-24T18:08:01.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.517e-06 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728852+0000", 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0090000000000001e-06 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728856+0000", 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728884+0000", 2024-04-24T18:08:01.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8602000000000002e-05 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728926+0000", 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1680000000000001e-05 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:01.728988+0000", 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2094999999999994e-05 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:01.724 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 1 2024-04-24T18:08:01.725 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:01.731 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:01.773 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #10, image image_client.0-clone10 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent11 2024-04-24T18:08:01.777 INFO:teuthology.orchestra.run.smithi049.stdout:1568 read 0x6698e8 thru 0x67589c (0xbfb5 bytes) 2024-04-24T18:08:01.778 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:01.786+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:01.778 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:01.779 INFO:teuthology.orchestra.run.smithi049.stdout:1573 writesame 0x390222e thru 0x390554d data_size 0x10(0x3320 bytes) 2024-04-24T18:08:01.785 INFO:teuthology.orchestra.run.smithi049.stdout:1577 read 0x12c14c4 thru 0x12c1d49 (0x886 bytes) 2024-04-24T18:08:01.786 INFO:teuthology.orchestra.run.smithi049.stdout:1578 read 0x2303042 thru 0x230b0a0 (0x805f bytes) 2024-04-24T18:08:01.786 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:01.786 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:01.786 INFO:teuthology.orchestra.run.smithi049.stdout:1582 punch from 0x54fd878 to 0x54fe28e, (0xa16 bytes) 2024-04-24T18:08:01.800 INFO:teuthology.orchestra.run.smithi049.stdout:1583 write 0xd940e9c thru 0xd94a223 (0x9388 bytes) 2024-04-24T18:08:01.803 INFO:teuthology.orchestra.run.smithi049.stdout:1584 writesame 0x934ba73 thru 0x9352ba1 data_size 0x3d(0x712f bytes) 2024-04-24T18:08:01.807 INFO:teuthology.orchestra.run.smithi049.stdout:1585 clone 13 order 20 su 32768 sc 3 2024-04-24T18:08:01.869 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:01.870 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:01.870 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:01.870 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:01.870 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:01.879 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:01.887+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:01.880 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:01.980 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:01.988+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31810 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 2920448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3659700360000002, 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052993206000000001, 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.051 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31810 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:02.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691739+0000", 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691742+0000", 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2800000000000002e-06 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691747+0000", 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4319999999999998e-06 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691750+0000", 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.247e-06 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744520+0000", 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052769498999999997 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744623+0000", 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000102808 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744733+0000", 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110282 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3523852759999997, 2024-04-24T18:08:02.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:08:02.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32297 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3233792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.060 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3400111930000005, 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052981070999999998, 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32297 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717698+0000", 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717705+0000", 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5529999999999998e-06 2024-04-24T18:08:02.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717711+0000", 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3849999999999999e-06 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717715+0000", 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770506+0000", 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052791267000000003 2024-04-24T18:08:02.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770587+0000", 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0783000000000002e-05 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770680+0000", 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2497000000000006e-05 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32299 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:02.064 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.33999019, 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053161606, 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32299 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717719+0000", 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717722+0000", 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.43e-06 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717729+0000", 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.576e-06 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717733+0000", 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0119999999999999e-06 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770707+0000", 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052974083999999998 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770791+0000", 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3418000000000007e-05 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770881+0000", 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0490999999999998e-05 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32301 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.068 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3399723199999993, 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053336189999999999, 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32301 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717737+0000", 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717740+0000", 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4090000000000001e-06 2024-04-24T18:08:02.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717746+0000", 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9989999999999997e-06 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717752+0000", 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0349999999999998e-06 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770915+0000", 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053162938 2024-04-24T18:08:02.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770989+0000", 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4567999999999995e-05 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771074+0000", 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4598000000000006e-05 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.072 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32303 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3270656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3398592419999993, 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053426976000000001, 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32303 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717850+0000", 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717854+0000", 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0509999999999998e-06 2024-04-24T18:08:02.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717863+0000", 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.346e-06 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717877+0000", 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3559e-05 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771102+0000", 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053224869000000001 2024-04-24T18:08:02.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771184+0000", 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2564000000000004e-05 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771278+0000", 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3178999999999997e-05 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32305 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3278848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3398270740000005, 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053587411000000001, 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32305 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717882+0000", 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717885+0000", 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8679999999999999e-06 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717892+0000", 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717896+0000", 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9360000000000001e-06 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771305+0000", 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053409310000000002 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771373+0000", 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8417999999999994e-05 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771470+0000", 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6827e-05 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32307 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3289088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3398100329999991, 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053767057, 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32307 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717899+0000", 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717902+0000", 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5019999999999999e-06 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717908+0000", 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7699999999999998e-06 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717911+0000", 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4979999999999998e-06 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771498+0000", 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053586747999999997 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771571+0000", 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3168000000000001e-05 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771667+0000", 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5810000000000006e-05 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32311 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3307520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3385136479999993, 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053356705999999997, 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32311 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719199+0000", 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.88e-06 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719205+0000", 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719209+0000", 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0749999999999999e-06 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772008+0000", 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052798572000000002 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772048+0000", 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0238999999999999e-05 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772553+0000", 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050474099999999996 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32313 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3315712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3384955900000008, 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053105309000000003, 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32313 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719216+0000", 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719223+0000", 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3010000000000003e-06 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719226+0000", 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7610000000000001e-06 2024-04-24T18:08:02.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772130+0000", 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052903238999999998 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772209+0000", 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9486000000000004e-05 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772320+0000", 2024-04-24T18:08:02.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011052099999999999 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32315 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3334144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3384789470000005, 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053250991999999997, 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32315 2024-04-24T18:08:02.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719233+0000", 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1789999999999998e-06 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719240+0000", 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5689999999999999e-06 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719244+0000", 2024-04-24T18:08:02.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5870000000000001e-06 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772345+0000", 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053100972000000003 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772413+0000", 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8071000000000006e-05 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772482+0000", 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8950999999999998e-05 2024-04-24T18:08:02.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32317 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.338396886, 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053370316000000001, 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32317 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719312+0000", 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719316+0000", 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9400000000000004e-06 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719326+0000", 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2439999999999999e-06 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719331+0000", 2024-04-24T18:08:02.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7049999999999996e-06 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772583+0000", 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053251860999999998 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772628+0000", 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5289e-05 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772683+0000", 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4882000000000001e-05 2024-04-24T18:08:02.100 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32319 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3352576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3383735429999994, 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053459060000000003, 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32319 2024-04-24T18:08:02.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719338+0000", 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5490000000000001e-06 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719346+0000", 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5059999999999998e-06 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719350+0000", 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1999999999999996e-06 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772709+0000", 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053359182999999998 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772748+0000", 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8654e-05 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772795+0000", 2024-04-24T18:08:02.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7437e-05 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32321 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3360768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3383553329999991, 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053627222000000002, 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32321 2024-04-24T18:08:02.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719354+0000", 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719361+0000", 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7449999999999998e-06 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719367+0000", 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3940000000000001e-06 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719378+0000", 2024-04-24T18:08:02.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1204e-05 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772830+0000", 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053451920999999999 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772896+0000", 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5623000000000005e-05 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772982+0000", 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5729000000000005e-05 2024-04-24T18:08:02.108 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32327 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3389440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3373391249999997, 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053038750000000003, 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32327 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720370+0000", 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720373+0000", 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5830000000000001e-06 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720380+0000", 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1550000000000001e-06 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720384+0000", 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0260000000000001e-06 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773322+0000", 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052937495000000001 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773364+0000", 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2561999999999997e-05 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773409+0000", 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5333000000000001e-05 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32329 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3397632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3373214260000008, 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053169842000000002, 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32329 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720391+0000", 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7779999999999999e-06 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720396+0000", 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6729999999999997e-06 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720400+0000", 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7179999999999998e-06 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773435+0000", 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053035169 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773497+0000", 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1715000000000001e-05 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773558+0000", 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1171000000000001e-05 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32331 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3416064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3373052970000003, 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053284320000000003, 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.117 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32331 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720404+0000", 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720406+0000", 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3190000000000002e-06 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720412+0000", 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8359999999999997e-06 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720416+0000", 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000004e-06 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773583+0000", 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053167076000000001 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773638+0000", 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105999999999997e-05 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773689+0000", 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0521e-05 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32333 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3424256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3372876849999997, 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053395479000000003, 2024-04-24T18:08:02.121 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32333 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720424+0000", 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720430+0000", 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720434+0000", 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8210000000000003e-06 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773721+0000", 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053287297999999997 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773762+0000", 2024-04-24T18:08:02.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1245000000000002e-05 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773818+0000", 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5585999999999999e-05 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32388 4.3 4:c61fab1f:::rbd_object_map.117432c990a0.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3351854559999996, 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.055543581000000002, 2024-04-24T18:08:02.125 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32388 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722527+0000", 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0319999999999999e-06 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722535+0000", 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9170000000000006e-06 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722539+0000", 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9709999999999998e-06 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774649+0000", 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052110109000000002 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774719+0000", 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9562999999999998e-05 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775050+0000", 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000331078 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777149+0000", 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020996639999999998 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:02.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777996+0000", 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084650899999999996 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778027+0000", 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0818999999999997e-05 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778068+0000", 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1338000000000003e-05 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4399104700000001, 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:08:02.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:08:02.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:08:02.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.136 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.136 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.136 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:02.137 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:02.138 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:02.145+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:02.238 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:02.246+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:02.308 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:02.308 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:02.308 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:02.308 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:02.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:02.359 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:02.367+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:02.440 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:02.448+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:02.487 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:02.487 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:02.487 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:02.487 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:02.487 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:02.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:02.541 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:02.548+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:02.641 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:02.649+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:02.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40940 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9800696389999999, 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036446457000000002, 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40940 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:02.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707548+0000", 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.251e-06 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707551+0000", 2024-04-24T18:08:02.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0019999999999998e-06 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707554+0000", 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743860+0000", 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036306756000000003 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743915+0000", 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4423999999999999e-05 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743994+0000", 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9207000000000002e-05 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40941 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1490944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9800360880000003, 2024-04-24T18:08:02.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036634427999999997, 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40941 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707583+0000", 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2089999999999999e-06 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707587+0000", 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1670000000000001e-06 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707590+0000", 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.554e-06 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744033+0000", 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036443542000000002 2024-04-24T18:08:02.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744112+0000", 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8578999999999994e-05 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744215+0000", 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103607 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.689 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40942 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1523712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9800250420000003, 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036756906999999998, 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40942 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707594+0000", 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7090000000000001e-06 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707596+0000", 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707598+0000", 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9149999999999999e-06 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744245+0000", 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036646570000000003 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744295+0000", 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0343000000000002e-05 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744349+0000", 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3661e-05 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.693 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40943 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9800163629999998, 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036956462000000002, 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40943 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:02.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707600+0000", 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707602+0000", 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.215e-06 2024-04-24T18:08:02.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707605+0000", 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0429999999999999e-06 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707607+0000", 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-06 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744380+0000", 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036773477999999998 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744464+0000", 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4048000000000006e-05 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744557+0000", 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3043999999999999e-05 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40944 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1589248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.697 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9800083629999996, 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037054089999999998, 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40944 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:02.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707610+0000", 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1319999999999999e-06 2024-04-24T18:08:02.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707613+0000", 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707614+0000", 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8819999999999999e-06 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744569+0000", 2024-04-24T18:08:02.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036954539000000002 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744613+0000", 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3671999999999998e-05 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744663+0000", 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0145000000000002e-05 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40945 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9800004629999997, 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037248315999999997, 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40945 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707616+0000", 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707620+0000", 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1999999999999999e-06 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707623+0000", 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1669999999999999e-06 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707625+0000", 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1110000000000002e-06 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744693+0000", 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037068564999999998 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744773+0000", 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9172999999999996e-05 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744865+0000", 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2305000000000005e-05 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.705 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40946 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799899199999995, 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037340299, 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40946 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707628+0000", 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2699999999999999e-06 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707631+0000", 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9519999999999999e-06 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707633+0000", 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8619999999999999e-06 2024-04-24T18:08:02.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744876+0000", 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037242550999999999 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744918+0000", 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2546000000000002e-05 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744967+0000", 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9320999999999998e-05 2024-04-24T18:08:02.709 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40947 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.979981864, 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037537860999999999, 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40947 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707636+0000", 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.186e-06 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707639+0000", 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707641+0000", 2024-04-24T18:08:02.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.86e-06 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744997+0000", 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037356364000000003 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745076+0000", 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8355000000000005e-05 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745173+0000", 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7379000000000004e-05 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40948 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1720320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799739399999998, 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037655182000000002, 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40948 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707644+0000", 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.269e-06 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707647+0000", 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9359999999999999e-06 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707649+0000", 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.815e-06 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745200+0000", 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037551189999999998 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745246+0000", 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6214999999999997e-05 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745298+0000", 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1960999999999999e-05 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40949 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.718 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799659109999999, 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037846454000000002, 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40949 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707652+0000", 2024-04-24T18:08:02.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.13e-06 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707655+0000", 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707658+0000", 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5770000000000001e-06 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745329+0000", 2024-04-24T18:08:02.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037671351999999998 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745408+0000", 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745498+0000", 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0043000000000005e-05 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40950 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799572310000002, 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037946947000000002, 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40950 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707661+0000", 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.15e-06 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707664+0000", 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.965e-06 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707666+0000", 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745508+0000", 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037841977999999998 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745555+0000", 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7098999999999997e-05 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745607+0000", 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1591999999999997e-05 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.726 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40951 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799487510000002, 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038137931999999999, 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40951 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707670+0000", 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7010000000000001e-06 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707673+0000", 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9119999999999998e-06 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707675+0000", 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9429999999999999e-06 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745637+0000", 2024-04-24T18:08:02.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037962379999999997 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745715+0000", 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7488000000000001e-05 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745806+0000", 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1723000000000003e-05 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40952 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799401249999997, 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038236762000000001, 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40952 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707678+0000", 2024-04-24T18:08:02.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.093e-06 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707681+0000", 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707683+0000", 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.951e-06 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745817+0000", 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038133854000000002 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745862+0000", 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5027000000000001e-05 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745914+0000", 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2120999999999998e-05 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.734 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40953 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799320920000003, 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038424458000000002, 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40953 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707686+0000", 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.567e-06 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707689+0000", 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9220000000000001e-06 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707691+0000", 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9369999999999998e-06 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745943+0000", 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038252047999999997 2024-04-24T18:08:02.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746018+0000", 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4938000000000007e-05 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746109+0000", 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1255000000000006e-05 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.738 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40954 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1916928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799235829999997, 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038528139000000003, 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40954 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:02.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707693+0000", 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:02.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.065e-06 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707697+0000", 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.881e-06 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707699+0000", 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.88e-06 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746120+0000", 2024-04-24T18:08:02.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038420937000000002 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746171+0000", 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0555e-05 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746222+0000", 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1025e-05 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40955 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799157630000002, 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718855000000003, 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40955 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707702+0000", 2024-04-24T18:08:02.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.212e-06 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707706+0000", 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.534e-06 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707708+0000", 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746254+0000", 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038545870000000003 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746331+0000", 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7805000000000005e-05 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746420+0000", 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8721000000000007e-05 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40956 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.746 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9799071450000003, 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038823347000000001, 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40956 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707711+0000", 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1659999999999999e-06 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707714+0000", 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.937e-06 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707716+0000", 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9920000000000002e-06 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746435+0000", 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718869000000003 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746482+0000", 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6882999999999998e-05 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746533+0000", 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1711e-05 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.750 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40957 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9798988709999996, 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039015052000000001, 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40957 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707721+0000", 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3110000000000001e-06 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707725+0000", 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2669999999999998e-06 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707727+0000", 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746563+0000", 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038836751000000003 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746641+0000", 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7900999999999998e-05 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746733+0000", 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2015999999999994e-05 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.754 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40958 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2048000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9798881780000004, 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039110399999999997, 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40958 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.755 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707730+0000", 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2640000000000001e-06 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707733+0000", 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9170000000000002e-06 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707735+0000", 2024-04-24T18:08:02.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746744+0000", 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039008958000000003 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746789+0000", 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5019999999999999e-05 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746839+0000", 2024-04-24T18:08:02.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0531999999999997e-05 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40959 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9798800050000001, 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039290798000000002, 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:02.759 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40959 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707738+0000", 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.24e-06 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707742+0000", 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6200000000000001e-06 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707744+0000", 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.615e-06 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746859+0000", 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039115095000000003 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746938+0000", 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:02.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.747028+0000", 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9742999999999998e-05 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:02.763 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:02.765 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:02.765 INFO:teuthology.orchestra.run.smithi049.stdout:flattening image image_client.0-clone12 2024-04-24T18:08:02.765 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:02.772+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:02.865 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:02.873+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:02.924 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #11, image image_client.0-clone11 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent12 2024-04-24T18:08:02.934 INFO:teuthology.orchestra.run.smithi049.stdout:1586 writesame 0x5cc6 thru 0xf2c5 data_size 0x50(0x9600 bytes) 2024-04-24T18:08:02.946 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:02.947 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:02.947 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:02.947 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:02.953 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:02.953 INFO:teuthology.orchestra.run.smithi049.stdout:1590 writesame 0xe10017 thru 0xe1cb3b data_size 0x5(0xcb25 bytes) 2024-04-24T18:08:02.958 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:02.961 INFO:teuthology.orchestra.run.smithi049.stdout:1593 write 0x90e199b thru 0x90e5656 (0x3cbc bytes) 2024-04-24T18:08:02.964 INFO:teuthology.orchestra.run.smithi049.stdout:1595 write 0x14f62f7 thru 0x150305b (0xcd65 bytes) 2024-04-24T18:08:02.965 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:02.973+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:02.973 INFO:teuthology.orchestra.run.smithi049.stdout:1596 read 0xb15ea8a thru 0xb16858f (0x9b06 bytes) 2024-04-24T18:08:02.973 INFO:teuthology.orchestra.run.smithi049.stdout:1597 punch from 0x7e0dcf6 to 0x7e16a69, (0x8d73 bytes) 2024-04-24T18:08:02.984 INFO:teuthology.orchestra.run.smithi049.stdout:1598 trunc from 0xd94a224 to 0x41c0805 2024-04-24T18:08:02.999 INFO:teuthology.orchestra.run.smithi049.stdout:1599 clone 14 order 21 su 131072 sc 9 2024-04-24T18:08:03.066 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:03.074+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:03.093 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:03.093 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:03.093 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:03.094 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:03.094 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:03.103 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:03.187 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:03.195+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:03.268 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:03.275+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:03.273 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6850 4.1c 4:3d3e8d61:::rbd_data.11749ffb1524.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.118841+0000", 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.161895247, 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066472299999999996, 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6850 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118841+0000", 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118841+0000", 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118839+0000", 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118848+0000", 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1929999999999999e-06 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118867+0000", 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8147000000000001e-05 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118875+0000", 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9840000000000001e-06 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118915+0000", 2024-04-24T18:08:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0797999999999998e-05 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118974+0000", 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8913999999999999e-05 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.120364+0000", 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001389794 2024-04-24T18:08:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125153+0000", 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047890989999999998 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125430+0000", 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027735900000000003 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125462+0000", 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1741e-05 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125488+0000", 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6044e-05 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33424 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.779002+0000", 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.501734073, 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065180560000000004, 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33424 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779002+0000", 2024-04-24T18:08:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779002+0000", 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778999+0000", 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999971 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779245+0000", 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00024558499999999999 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779267+0000", 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1894000000000001e-05 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779275+0000", 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3210000000000005e-06 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779336+0000", 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0927000000000001e-05 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779427+0000", 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1032000000000005e-05 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779666+0000", 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000238874 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.782980+0000", 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033137829999999998 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785422+0000", 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024418399999999998 2024-04-24T18:08:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785482+0000", 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9886999999999998e-05 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785520+0000", 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8686000000000002e-05 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33425 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:head [delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.779290+0000", 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.501446457, 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006444094, 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33425 2024-04-24T18:08:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779290+0000", 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779290+0000", 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779289+0000", 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779293+0000", 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8879999999999999e-06 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779306+0000", 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3115999999999999e-05 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779311+0000", 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4719999999999997e-06 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.780818+0000", 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001506949 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.780904+0000", 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6007999999999994e-05 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.781092+0000", 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018842300000000001 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.783464+0000", 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023717180000000001 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785650+0000", 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002186571 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785695+0000", 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4993000000000001e-05 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785734+0000", 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8615999999999998e-05 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35123 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 196608~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058125+0000", 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222611512, 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065029679999999996, 2024-04-24T18:08:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35123 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058125+0000", 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058125+0000", 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058124+0000", 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058127+0000", 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1320000000000001e-06 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058132+0000", 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7350000000000001e-06 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058136+0000", 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4929999999999998e-06 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.063278+0000", 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051423079999999999 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.063355+0000", 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6891000000000006e-05 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064628+0000", 2024-04-24T18:08:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012727089999999999 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35124 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222596373, 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066858179999999996, 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35124 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058142+0000", 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1689999999999999e-06 2024-04-24T18:08:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058147+0000", 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6659999999999996e-06 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058151+0000", 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064658+0000", 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065067579999999996 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064740+0000", 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2124000000000002e-05 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064826+0000", 2024-04-24T18:08:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5892999999999999e-05 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35125 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 378880~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058161+0000", 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222575566, 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006851673, 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35125 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058161+0000", 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058161+0000", 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058160+0000", 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058163+0000", 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4789999999999999e-06 2024-04-24T18:08:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058169+0000", 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4489999999999996e-06 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058173+0000", 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1559999999999997e-06 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064853+0000", 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066795270000000002 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064929+0000", 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6259000000000003e-05 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065012+0000", 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3146999999999995e-05 2024-04-24T18:08:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35126 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 471040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058178+0000", 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222558512999999, 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0070301249999999999, 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35126 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058178+0000", 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058178+0000", 2024-04-24T18:08:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058177+0000", 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058180+0000", 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.339e-06 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058185+0000", 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2950000000000001e-06 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058189+0000", 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.455e-06 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065040+0000", 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0068511880000000002 2024-04-24T18:08:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065117+0000", 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6890000000000004e-05 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065208+0000", 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1278000000000003e-05 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35266 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 18432~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222543352000001, 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0072003559999999998, 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35266 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058195+0000", 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.334e-06 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058201+0000", 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.37e-06 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058205+0000", 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5619999999999999e-06 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065235+0000", 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0070298649999999997 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065310+0000", 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5741000000000005e-05 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065393+0000", 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2822000000000001e-05 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35267 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 110592~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222526864000001, 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073675850000000003, 2024-04-24T18:08:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35267 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058212+0000", 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4590000000000001e-06 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058217+0000", 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9669999999999998e-06 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058221+0000", 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.811e-06 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065419+0000", 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071978709999999998 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065494+0000", 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5224999999999999e-05 2024-04-24T18:08:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065577+0000", 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2609999999999997e-05 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35268 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 202752~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058226+0000", 2024-04-24T18:08:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222510575999999, 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0075397320000000004, 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35268 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058226+0000", 2024-04-24T18:08:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058226+0000", 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058225+0000", 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058228+0000", 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058233+0000", 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6550000000000001e-06 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058237+0000", 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6950000000000002e-06 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065604+0000", 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007366877 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065683+0000", 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9515e-05 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065765+0000", 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2191e-05 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35269 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 294912~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222494857999999, 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0077089660000000003, 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35269 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058243+0000", 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3e-06 2024-04-24T18:08:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058249+0000", 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8320000000000002e-06 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058253+0000", 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8759999999999998e-06 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065792+0000", 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0075390250000000004 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065868+0000", 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5660000000000004e-05 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065950+0000", 2024-04-24T18:08:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2601000000000004e-05 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35270 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222478393999999, 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0078762390000000002, 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35270 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058260+0000", 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4080000000000001e-06 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058266+0000", 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8889999999999999e-06 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058270+0000", 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7220000000000001e-06 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065977+0000", 2024-04-24T18:08:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0077071939999999997 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066052+0000", 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5035000000000002e-05 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066134+0000", 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2338999999999999e-05 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35271 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 477184~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058274+0000", 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 10.222462532, 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081649119999999999, 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35271 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058274+0000", 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058274+0000", 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058273+0000", 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058276+0000", 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0549999999999998e-06 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058281+0000", 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7239999999999997e-06 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058285+0000", 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5310000000000002e-06 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066237+0000", 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0079519829999999993 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066333+0000", 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6749999999999994e-05 2024-04-24T18:08:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066439+0000", 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105169 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37455 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.795782+0000", 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.4849539469999993, 2024-04-24T18:08:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053832250000000002, 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37455 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795782+0000", 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795782+0000", 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795781+0000", 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795789+0000", 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3159999999999999e-06 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795801+0000", 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2840000000000001e-05 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795808+0000", 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0899999999999999e-06 2024-04-24T18:08:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797600+0000", 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017916620000000001 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797703+0000", 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010255 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.798106+0000", 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00040297299999999999 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801065+0000", 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0029588520000000001 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801072+0000", 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3699999999999997e-06 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801120+0000", 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7842e-05 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801166+0000", 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5840000000000002e-05 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37458 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.4837760979999999, 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.005658667, 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37458 2024-04-24T18:08:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796964+0000", 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3640000000000001e-06 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796972+0000", 2024-04-24T18:08:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0369999999999995e-06 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796977+0000", 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0629999999999996e-06 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.800493+0000", 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035157029999999998 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.800539+0000", 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6363000000000003e-05 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.800749+0000", 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000209614 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802040+0000", 2024-04-24T18:08:03.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001291726 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802530+0000", 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00048976499999999997 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802581+0000", 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0992999999999998e-05 2024-04-24T18:08:03.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802619+0000", 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7693e-05 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37459 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.797109+0000", 2024-04-24T18:08:03.355 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.4836274300000003, 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0072136969999999998, 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37459 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797109+0000", 2024-04-24T18:08:03.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797109+0000", 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797108+0000", 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797114+0000", 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0270000000000001e-06 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797125+0000", 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1119e-05 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797134+0000", 2024-04-24T18:08:03.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4120000000000001e-06 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801288+0000", 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041545030000000004 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801362+0000", 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3532e-05 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801660+0000", 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00029880400000000001 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802752+0000", 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001091369 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.804257+0000", 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015050910000000001 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.804295+0000", 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8139999999999997e-05 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.804323+0000", 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7572e-05 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.361 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:37548 4.19 e43/26 4:9d690b33:::rbd_data.1174b285a870.0000000000000001:head v 43'55, mlcod=42'53)", 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.843919+0000", 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.4368176960000003, 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0090564110000000003, 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.843919+0000", 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.843919+0000", 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.843917+0000", 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.844810+0000", 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089266599999999997 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.844974+0000", 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000163859 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.844980+0000", 2024-04-24T18:08:03.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9660000000000001e-06 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845069+0000", 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9844000000000002e-05 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845090+0000", 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0069e-05 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.852929+0000", 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0078399070000000001 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.852975+0000", 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5578999999999998e-05 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37550 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.845638+0000", 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.4350978889999997, 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0079207649999999998, 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37550 2024-04-24T18:08:03.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845638+0000", 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845638+0000", 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845636+0000", 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999976 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845645+0000", 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1160000000000005e-06 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845657+0000", 2024-04-24T18:08:03.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1799e-05 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845664+0000", 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845832+0000", 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000168233 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845934+0000", 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101734 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.846328+0000", 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00039441699999999999 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.847343+0000", 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010148819999999999 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.853505+0000", 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061618580000000001 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.853537+0000", 2024-04-24T18:08:03.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2351999999999998e-05 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.853559+0000", 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2112000000000001e-05 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40434 4.1c 4:3ae45a05:::rbd_data.1174f9704cae.0000000000000006:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:03.373 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.696803+0000", 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.5839334349999996, 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071709690000000001, 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40434 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696803+0000", 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696803+0000", 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696801+0000", 2024-04-24T18:08:03.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696809+0000", 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0080000000000002e-06 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696831+0000", 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1225000000000001e-05 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696840+0000", 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6490000000000003e-06 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696893+0000", 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2512999999999997e-05 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696980+0000", 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6694999999999996e-05 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.697173+0000", 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019387399999999999 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.699556+0000", 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023828569999999999 2024-04-24T18:08:03.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.703880+0000", 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043240190000000001 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.703933+0000", 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2735999999999998e-05 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.703974+0000", 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0822999999999999e-05 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:42560 4.1c 4:3c3da860:::rbd_data.1174578192f5.0000000000000014:head [stat,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:58.250773+0000", 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.0299628869999999, 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0091032000000000005, 2024-04-24T18:08:03.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 42560 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250773+0000", 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250773+0000", 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250772+0000", 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250780+0000", 2024-04-24T18:08:03.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2099999999999993e-06 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250798+0000", 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.853e-05 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250806+0000", 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.712e-06 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:03.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250855+0000", 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8823000000000003e-05 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250935+0000", 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0557999999999997e-05 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.251072+0000", 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136775 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.251688+0000", 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061615400000000005 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.259778+0000", 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0080894409999999993 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.259837+0000", 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8857999999999998e-05 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:03.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.259877+0000", 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.994e-05 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:03.386 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:03.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:03.388 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:03.396+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:03.413 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:03.489 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:03.496+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:03.570 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:03.571 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:03.571 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:03.571 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:03.581 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:03.584 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:03.603 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:03.603 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:03.611+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:03.623 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:03.644 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:08:03.650 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:03.673 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:03.690 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:03.697+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:03.696 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:03.703 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone13 from 0x41c0805 (overlap 0x41c0805) to 0x2773803 2024-04-24T18:08:03.720 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:03.730 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:03.730 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:03.742 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:03.743 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:03.745 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:03.746 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:03.790 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:03.798+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:03.807 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #12, image image_client.0-clone12 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent13 2024-04-24T18:08:03.887 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:03.887 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:03.891 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:03.900+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:03.898 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:03.992 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:04.001+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:04.034 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:04.034 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:04.044 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:04.062 INFO:teuthology.orchestra.run.smithi049.stdout:1600 trunc from 0x41c0805 to 0xea9b385 2024-04-24T18:08:04.071 INFO:teuthology.orchestra.run.smithi049.stdout:1601 read 0x175bb2d thru 0x176b865 (0xfd39 bytes) 2024-04-24T18:08:04.071 INFO:teuthology.orchestra.run.smithi049.stdout:1602 read 0xdc523d0 thru 0xdc5ce05 (0xaa36 bytes) 2024-04-24T18:08:04.071 INFO:teuthology.orchestra.run.smithi049.stdout:1603 writesame 0xc570a1b thru 0xc579113 data_size 0x1(0x86f9 bytes) 2024-04-24T18:08:04.090 INFO:teuthology.orchestra.run.smithi049.stdout:1604 read 0x168f454 thru 0x169ecdc (0xf889 bytes) 2024-04-24T18:08:04.091 INFO:teuthology.orchestra.run.smithi049.stdout:1605 read 0xd585463 thru 0xd58cec3 (0x7a61 bytes) 2024-04-24T18:08:04.091 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.091 INFO:teuthology.orchestra.run.smithi049.stdout:1607 read 0x325bcd1 thru 0x325cb6a (0xe9a bytes) 2024-04-24T18:08:04.092 INFO:teuthology.orchestra.run.smithi049.stdout:1609 read 0xc613f0c thru 0xc61f207 (0xb2fc bytes) 2024-04-24T18:08:04.092 INFO:teuthology.orchestra.run.smithi049.stdout:1611 punch from 0xae6f64 to 0xaf234c, (0xb3e8 bytes) 2024-04-24T18:08:04.092 INFO:teuthology.orchestra.run.smithi049.stdout:1612 read 0x64f84d7 thru 0x65020c0 (0x9bea bytes) 2024-04-24T18:08:04.092 INFO:teuthology.orchestra.run.smithi049.stdout:1613 writesame 0x382c233 thru 0x3834472 data_size 0x20(0x8240 bytes) 2024-04-24T18:08:04.102 INFO:teuthology.orchestra.run.smithi049.stdout:1616 trunc from 0xea9b385 to 0x8ab8496 2024-04-24T18:08:04.113 INFO:teuthology.orchestra.run.smithi049.stdout:1618 trunc from 0x8ab8496 to 0xa097eda 2024-04-24T18:08:04.119 INFO:teuthology.orchestra.run.smithi049.stdout:1620 read 0x72b256f thru 0x72b7785 (0x5217 bytes) 2024-04-24T18:08:04.119 INFO:teuthology.orchestra.run.smithi049.stdout:1622 read 0x8ceef6b thru 0x8cf837b (0x9411 bytes) 2024-04-24T18:08:04.127 INFO:teuthology.orchestra.run.smithi049.stdout:1623 write 0xc94fa81 thru 0xc95d603 (0xdb83 bytes) 2024-04-24T18:08:04.128 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:04.137+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:04.130 INFO:teuthology.orchestra.run.smithi049.stdout:1624 writesame 0x3c69f3 thru 0x3cfd83 data_size 0x1(0x9391 bytes) 2024-04-24T18:08:04.161 INFO:teuthology.orchestra.run.smithi049.stdout:1626 writesame 0xb51bcc0 thru 0xb521369 data_size 0x1(0x56aa bytes) 2024-04-24T18:08:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:1627 punch from 0x26ff1c1 to 0x2706774, (0x75b3 bytes) 2024-04-24T18:08:04.173 INFO:teuthology.orchestra.run.smithi049.stdout:1629 writesame 0xb895d45 thru 0xb89753c data_size 0x8(0x17f8 bytes) 2024-04-24T18:08:04.183 INFO:teuthology.orchestra.run.smithi049.stdout:1631 trunc from 0xc95d604 to 0x495e2b3 2024-04-24T18:08:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:1632 punch from 0x5dcd67 to 0x5dffe9, (0x3282 bytes) 2024-04-24T18:08:04.192 INFO:teuthology.orchestra.run.smithi049.stdout:1633 read 0x25a399e thru 0x25a84d5 (0x4b38 bytes) 2024-04-24T18:08:04.193 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:04.201+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:04.200 INFO:teuthology.orchestra.run.smithi049.stdout:1634 write 0x6f334a8 thru 0x6f400e1 (0xcc3a bytes) 2024-04-24T18:08:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.203 INFO:teuthology.orchestra.run.smithi049.stdout:1636 read 0x4f2a562 thru 0x4f2f778 (0x5217 bytes) 2024-04-24T18:08:04.204 INFO:teuthology.orchestra.run.smithi049.stdout:1637 write 0x18eeb3f thru 0x18fe48d (0xf94f bytes) 2024-04-24T18:08:04.208 INFO:teuthology.orchestra.run.smithi049.stdout:1638 punch from 0x2be9cd1 to 0x2bf91d7, (0xf506 bytes) 2024-04-24T18:08:04.209 INFO:teuthology.orchestra.run.smithi049.stdout:1639 read 0x19eac51 thru 0x19f70a1 (0xc451 bytes) 2024-04-24T18:08:04.219 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:04.219 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:04.221 INFO:teuthology.orchestra.run.smithi049.stdout:1640 write 0xc6413e9 thru 0xc64c64c (0xb264 bytes) 2024-04-24T18:08:04.225 INFO:teuthology.orchestra.run.smithi049.stdout:1641 writesame 0x35796f0 thru 0x3580590 data_size 0x1(0x6ea1 bytes) 2024-04-24T18:08:04.231 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:1642 read 0x15b83db thru 0x15c59cb (0xd5f1 bytes) 2024-04-24T18:08:04.242 INFO:teuthology.orchestra.run.smithi049.stdout:1643 write 0x5946c12 thru 0x594e1b9 (0x75a8 bytes) 2024-04-24T18:08:04.245 INFO:teuthology.orchestra.run.smithi049.stdout:1644 read 0xb03d195 thru 0xb044ec1 (0x7d2d bytes) 2024-04-24T18:08:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:1645 punch from 0x360b035 to 0x360d864, (0x282f bytes) 2024-04-24T18:08:04.246 INFO:teuthology.orchestra.run.smithi049.stdout:1646 writesame 0x3fdb058 thru 0x3fe87cb data_size 0x2(0xd774 bytes) 2024-04-24T18:08:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:1648 punch from 0xae37ce8 to 0xae3b30b, (0x3623 bytes) 2024-04-24T18:08:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:1649 punch from 0x97e0e11 to 0x97e2d9a, (0x1f89 bytes) 2024-04-24T18:08:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.257 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.264 INFO:teuthology.orchestra.run.smithi049.stdout:1653 write 0xd5a3f94 thru 0xd5ac02c (0x8099 bytes) 2024-04-24T18:08:04.266 INFO:teuthology.orchestra.run.smithi049.stdout:1654 write 0x92c03ba thru 0x92cf7cf (0xf416 bytes) 2024-04-24T18:08:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:1656 punch from 0xc5a428 to 0xc5b2f7, (0xecf bytes) 2024-04-24T18:08:04.270 INFO:teuthology.orchestra.run.smithi049.stdout:1657 write 0x783f0a9 thru 0x784386a (0x47c2 bytes) 2024-04-24T18:08:04.271 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.276 INFO:teuthology.orchestra.run.smithi049.stdout:1659 trunc from 0xd5ac02d to 0x86f65b4 2024-04-24T18:08:04.293 INFO:teuthology.orchestra.run.smithi049.stdout:1660 trunc from 0x86f65b4 to 0x202512e 2024-04-24T18:08:04.294 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:04.302+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:04.330 INFO:teuthology.orchestra.run.smithi049.stdout:1661 trunc from 0x202512e to 0x56230ef 2024-04-24T18:08:04.336 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.338 INFO:teuthology.orchestra.run.smithi049.stdout:1666 trunc from 0x56230ef to 0x368f2d7 2024-04-24T18:08:04.343 INFO:teuthology.orchestra.run.smithi049.stdout:1667 read 0x151797b thru 0x151e8a0 (0x6f26 bytes) 2024-04-24T18:08:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:1668 trunc from 0x368f2d7 to 0x9452913 2024-04-24T18:08:04.355 INFO:teuthology.orchestra.run.smithi049.stdout:1669 punch from 0x5b459ab to 0x5b4d17b, (0x77d0 bytes) 2024-04-24T18:08:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:1670 writesame 0xc7321c0 thru 0xc733602 data_size 0xd(0x1443 bytes) 2024-04-24T18:08:04.364 INFO:teuthology.orchestra.run.smithi049.stdout:1671 write 0x5a860d9 thru 0x5a93e3a (0xdd62 bytes) 2024-04-24T18:08:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:1672 read 0x3aa5ba thru 0x3b15c7 (0x700e bytes) 2024-04-24T18:08:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:1673 read 0x6924bcf thru 0x69286a0 (0x3ad2 bytes) 2024-04-24T18:08:04.367 INFO:teuthology.orchestra.run.smithi049.stdout:1674 writesame 0xa8e8c70 thru 0xa8f4bdc data_size 0x87(0xbf6d bytes) 2024-04-24T18:08:04.377 INFO:teuthology.orchestra.run.smithi049.stdout:1675 writesame 0xd428f10 thru 0xd42e659 data_size 0x1(0x574a bytes) 2024-04-24T18:08:04.378 INFO:teuthology.orchestra.run.smithi049.stdout:1676 punch from 0x585c315 to 0x5865858, (0x9543 bytes) 2024-04-24T18:08:04.378 INFO:teuthology.orchestra.run.smithi049.stdout:1678 writesame 0x5d5f4c8 thru 0x5d672d1 data_size 0x49(0x7e0a bytes) 2024-04-24T18:08:04.380 INFO:teuthology.orchestra.run.smithi049.stdout:1679 punch from 0x84d47af to 0x84ddf98, (0x97e9 bytes) 2024-04-24T18:08:04.380 INFO:teuthology.orchestra.run.smithi049.stdout:1680 write 0x9d07d4b thru 0x9d09839 (0x1aef bytes) 2024-04-24T18:08:04.383 INFO:teuthology.orchestra.run.smithi049.stdout:1681 read 0x8eac9b7 thru 0x8eba132 (0xd77c bytes) 2024-04-24T18:08:04.383 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.383 INFO:teuthology.orchestra.run.smithi049.stdout:1683 read 0x843298d thru 0x8438093 (0x5707 bytes) 2024-04-24T18:08:04.383 INFO:teuthology.orchestra.run.smithi049.stdout:1684 read 0x2641e48 thru 0x264ac3c (0x8df5 bytes) 2024-04-24T18:08:04.383 INFO:teuthology.orchestra.run.smithi049.stdout:1685 read 0xc857d56 thru 0xc865986 (0xdc31 bytes) 2024-04-24T18:08:04.390 INFO:teuthology.orchestra.run.smithi049.stdout:1686 trunc from 0xd42e65a to 0x61c7a9b 2024-04-24T18:08:04.395 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:04.403+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:04.405 INFO:teuthology.orchestra.run.smithi049.stdout:1687 trunc from 0x61c7a9b to 0x305ba73 2024-04-24T18:08:04.408 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:04.408 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:04.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:04.419 INFO:teuthology.orchestra.run.smithi049.stdout:1688 write 0x33ae84c thru 0x33b63c4 (0x7b79 bytes) 2024-04-24T18:08:04.422 INFO:teuthology.orchestra.run.smithi049.stdout:1689 read 0x2525496 thru 0x252776b (0x22d6 bytes) 2024-04-24T18:08:04.422 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.433 INFO:teuthology.orchestra.run.smithi049.stdout:1695 trunc from 0x33b63c5 to 0xd5a983c 2024-04-24T18:08:04.439 INFO:teuthology.orchestra.run.smithi049.stdout:1696 punch from 0x5d9d48 to 0x5e781a, (0xdad2 bytes) 2024-04-24T18:08:04.439 INFO:teuthology.orchestra.run.smithi049.stdout:1697 writesame 0x8dff6f5 thru 0x8e0ab68 data_size 0x2(0xb474 bytes) 2024-04-24T18:08:04.450 INFO:teuthology.orchestra.run.smithi049.stdout:1699 punch from 0x31b958c to 0x31c184e, (0x82c2 bytes) 2024-04-24T18:08:04.455 INFO:teuthology.orchestra.run.smithi049.stdout:1700 trunc from 0xd5a983c to 0x8aab36a 2024-04-24T18:08:04.462 INFO:teuthology.orchestra.run.smithi049.stdout:1701 punch from 0x22e4e7 to 0x239ed8, (0xb9f1 bytes) 2024-04-24T18:08:04.462 INFO:teuthology.orchestra.run.smithi049.stdout:1703 punch from 0x7369390 to 0x736a76c, (0x13dc bytes) 2024-04-24T18:08:04.466 INFO:teuthology.orchestra.run.smithi049.stdout:1705 trunc from 0x8aab36a to 0x9642961 2024-04-24T18:08:04.474 INFO:teuthology.orchestra.run.smithi049.stdout:1706 trunc from 0x9642961 to 0x3d7ab8a 2024-04-24T18:08:04.483 INFO:teuthology.orchestra.run.smithi049.stdout:1707 trunc from 0x3d7ab8a to 0x451f214 2024-04-24T18:08:04.487 INFO:teuthology.orchestra.run.smithi049.stdout:1708 punch from 0x22d9c08 to 0x22ded62, (0x515a bytes) 2024-04-24T18:08:04.488 INFO:teuthology.orchestra.run.smithi049.stdout:1709 read 0x33dce88 thru 0x33de1b6 (0x132f bytes) 2024-04-24T18:08:04.491 INFO:teuthology.orchestra.run.smithi049.stdout:1710 trunc from 0x451f214 to 0x166ba61 2024-04-24T18:08:04.496 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:04.504+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:04.520 INFO:teuthology.orchestra.run.smithi049.stdout:1711 write 0xc6dfbf3 thru 0xc6edaf0 (0xdefe bytes) 2024-04-24T18:08:04.525 INFO:teuthology.orchestra.run.smithi049.stdout:1712 read 0x2238a17 thru 0x223b64d (0x2c37 bytes) 2024-04-24T18:08:04.525 INFO:teuthology.orchestra.run.smithi049.stdout:1714 read 0x485f79a thru 0x4864681 (0x4ee8 bytes) 2024-04-24T18:08:04.526 INFO:teuthology.orchestra.run.smithi049.stdout:1715 trunc from 0xc6edaf1 to 0xa820084 2024-04-24T18:08:04.536 INFO:teuthology.orchestra.run.smithi049.stdout:1716 read 0x374f006 thru 0x3756a17 (0x7a12 bytes) 2024-04-24T18:08:04.536 INFO:teuthology.orchestra.run.smithi049.stdout:1717 write 0x4b09b thru 0x4c1b5 (0x111b bytes) 2024-04-24T18:08:04.544 INFO:teuthology.orchestra.run.smithi049.stdout:1718 write 0x14a4ed6 thru 0x14b2de2 (0xdf0d bytes) 2024-04-24T18:08:04.545 INFO:teuthology.orchestra.run.smithi049.stdout:1720 read 0x67676a5 thru 0x6773cae (0xc60a bytes) 2024-04-24T18:08:04.553 INFO:teuthology.orchestra.run.smithi049.stdout:1721 write 0xb913422 thru 0xb913ca2 (0x881 bytes) 2024-04-24T18:08:04.556 INFO:teuthology.orchestra.run.smithi049.stdout:1722 write 0x4178162 thru 0x4187b73 (0xfa12 bytes) 2024-04-24T18:08:04.560 INFO:teuthology.orchestra.run.smithi049.stdout:1723 write 0x539f8af thru 0x53a438f (0x4ae1 bytes) 2024-04-24T18:08:04.572 INFO:teuthology.orchestra.run.smithi049.stdout:1724 write 0xec307a0 thru 0xec38aa6 (0x8307 bytes) 2024-04-24T18:08:04.575 INFO:teuthology.orchestra.run.smithi049.stdout:1725 punch from 0x671509 to 0x672707, (0x11fe bytes) 2024-04-24T18:08:04.575 INFO:teuthology.orchestra.run.smithi049.stdout:1727 writesame 0x7c9d7ea thru 0x7ca88c6 data_size 0x1(0xb0dd bytes) 2024-04-24T18:08:04.591 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:04.591 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:04.595 INFO:teuthology.orchestra.run.smithi049.stdout:1728 read 0x32045bc thru 0x320689f (0x22e4 bytes) 2024-04-24T18:08:04.595 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.595 INFO:teuthology.orchestra.run.smithi049.stdout:1730 write 0x9494d11 thru 0x94a28d1 (0xdbc1 bytes) 2024-04-24T18:08:04.596 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:04.605+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:04.598 INFO:teuthology.orchestra.run.smithi049.stdout:1731 writesame 0x8fdf5cc thru 0x8febc5f data_size 0x47(0xc694 bytes) 2024-04-24T18:08:04.602 INFO:teuthology.orchestra.run.smithi049.stdout:1732 punch from 0x206b72b to 0x20766b3, (0xaf88 bytes) 2024-04-24T18:08:04.602 INFO:teuthology.orchestra.run.smithi049.stdout:1733 write 0xe1818f6 thru 0xe1840b3 (0x27be bytes) 2024-04-24T18:08:04.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:04.605 INFO:teuthology.orchestra.run.smithi049.stdout:1734 write 0x91ea5fd thru 0x91f9d0c (0xf710 bytes) 2024-04-24T18:08:04.607 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.608 INFO:teuthology.orchestra.run.smithi049.stdout:1738 read 0x77a5c91 thru 0x77a6f7e (0x12ee bytes) 2024-04-24T18:08:04.608 INFO:teuthology.orchestra.run.smithi049.stdout:1740 write 0x730626f thru 0x730e0e6 (0x7e78 bytes) 2024-04-24T18:08:04.613 INFO:teuthology.orchestra.run.smithi049.stdout:1743 trunc from 0xec38aa7 to 0xcab2dba 2024-04-24T18:08:04.622 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.622 INFO:teuthology.orchestra.run.smithi049.stdout:1745 write 0x5503156 thru 0x550bea6 (0x8d51 bytes) 2024-04-24T18:08:04.624 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.631 INFO:teuthology.orchestra.run.smithi049.stdout:1752 trunc from 0xcab2dba to 0x6f41fd5 2024-04-24T18:08:04.642 INFO:teuthology.orchestra.run.smithi049.stdout:1753 write 0x6af70f7 thru 0x6b06bbb (0xfac5 bytes) 2024-04-24T18:08:04.645 INFO:teuthology.orchestra.run.smithi049.stdout:1754 punch from 0x5b81276 to 0x5b87147, (0x5ed1 bytes) 2024-04-24T18:08:04.645 INFO:teuthology.orchestra.run.smithi049.stdout:1756 read 0x62cbd9b thru 0x62db796 (0xf9fc bytes) 2024-04-24T18:08:04.645 INFO:teuthology.orchestra.run.smithi049.stdout:1757 writesame 0x5d50e89 thru 0x5d5736f data_size 0x1(0x64e7 bytes) 2024-04-24T18:08:04.660 INFO:teuthology.orchestra.run.smithi049.stdout:1758 punch from 0x17e07a4 to 0x17eef83, (0xe7df bytes) 2024-04-24T18:08:04.660 INFO:teuthology.orchestra.run.smithi049.stdout:1759 writesame 0x5d86315 thru 0x5d86ece data_size 0x2(0xbba bytes) 2024-04-24T18:08:04.662 INFO:teuthology.orchestra.run.smithi049.stdout:1761 read 0x358e1fc thru 0x359c233 (0xe038 bytes) 2024-04-24T18:08:04.662 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.664 INFO:teuthology.orchestra.run.smithi049.stdout:1763 trunc from 0x6f41fd5 to 0x4a3b232 2024-04-24T18:08:04.675 INFO:teuthology.orchestra.run.smithi049.stdout:1765 read 0x775b30 thru 0x78117b (0xb64c bytes) 2024-04-24T18:08:04.692 INFO:teuthology.orchestra.run.smithi049.stdout:1766 write 0xe819953 thru 0xe824d10 (0xb3be bytes) 2024-04-24T18:08:04.696 INFO:teuthology.orchestra.run.smithi049.stdout:1767 write 0x8fbb60d thru 0x8fbe63e (0x3032 bytes) 2024-04-24T18:08:04.697 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:04.705+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:04.699 INFO:teuthology.orchestra.run.smithi049.stdout:1769 read 0x9294656 thru 0x929f592 (0xaf3d bytes) 2024-04-24T18:08:04.699 INFO:teuthology.orchestra.run.smithi049.stdout:1770 write 0x4722633 thru 0x4725f64 (0x3932 bytes) 2024-04-24T18:08:04.700 INFO:teuthology.orchestra.run.smithi049.stdout:1771 read 0x8ebbec4 thru 0x8ec2498 (0x65d5 bytes) 2024-04-24T18:08:04.700 INFO:teuthology.orchestra.run.smithi049.stdout:1772 write 0xe54aa80 thru 0xe5584ad (0xda2e bytes) 2024-04-24T18:08:04.703 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.703 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.703 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.704 INFO:teuthology.orchestra.run.smithi049.stdout:1777 writesame 0x198bd51 thru 0x19963b6 data_size 0x3b(0xa666 bytes) 2024-04-24T18:08:04.705 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.705 INFO:teuthology.orchestra.run.smithi049.stdout:1780 write 0x9fade1d thru 0x9fb9e17 (0xbffb bytes) 2024-04-24T18:08:04.708 INFO:teuthology.orchestra.run.smithi049.stdout:1781 punch from 0x3379d0a to 0x3383b40, (0x9e36 bytes) 2024-04-24T18:08:04.708 INFO:teuthology.orchestra.run.smithi049.stdout:1782 punch from 0xba96bbb to 0xba9c535, (0x597a bytes) 2024-04-24T18:08:04.708 INFO:teuthology.orchestra.run.smithi049.stdout:1783 writesame 0xae89647 thru 0xae8e59c data_size 0xf(0x4f56 bytes) 2024-04-24T18:08:04.711 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.711 INFO:teuthology.orchestra.run.smithi049.stdout:1785 read 0x4f2e7a0 thru 0x4f3d3b6 (0xec17 bytes) 2024-04-24T18:08:04.711 INFO:teuthology.orchestra.run.smithi049.stdout:1786 read 0x56df273 thru 0x56ecbdf (0xd96d bytes) 2024-04-24T18:08:04.712 INFO:teuthology.orchestra.run.smithi049.stdout:1787 writesame 0x8d9ebf6 thru 0x8dab65f data_size 0xd(0xca6a bytes) 2024-04-24T18:08:04.717 INFO:teuthology.orchestra.run.smithi049.stdout:1788 trunc from 0xe824d11 to 0xdd58b5d 2024-04-24T18:08:04.723 INFO:teuthology.orchestra.run.smithi049.stdout:1789 write 0x862f0c4 thru 0x86314ca (0x2407 bytes) 2024-04-24T18:08:04.725 INFO:teuthology.orchestra.run.smithi049.stdout:1790 writesame 0x13d35b5 thru 0x13d8469 data_size 0x1(0x4eb5 bytes) 2024-04-24T18:08:04.736 INFO:teuthology.orchestra.run.smithi049.stdout:1791 write 0x4dbb2ab thru 0x4dc5c52 (0xa9a8 bytes) 2024-04-24T18:08:04.739 INFO:teuthology.orchestra.run.smithi049.stdout:1792 write 0x1233917 thru 0x123533f (0x1a29 bytes) 2024-04-24T18:08:04.740 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.740 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.741 INFO:teuthology.orchestra.run.smithi049.stdout:1796 read 0x4ba50fe thru 0x4bb3a19 (0xe91c bytes) 2024-04-24T18:08:04.744 INFO:teuthology.orchestra.run.smithi049.stdout:1797 trunc from 0xdd58b5d to 0xac90120 2024-04-24T18:08:04.758 INFO:teuthology.orchestra.run.smithi049.stdout:1799 trunc from 0xac90120 to 0x815b98f 2024-04-24T18:08:04.770 INFO:teuthology.orchestra.run.smithi049.stdout:1800 write 0x52c14d7 thru 0x52c8aca (0x75f4 bytes) 2024-04-24T18:08:04.771 INFO:teuthology.orchestra.run.smithi049.stdout:1802 writesame 0x17b8866 thru 0x17ba0a9 data_size 0x2(0x1844 bytes) 2024-04-24T18:08:04.774 INFO:teuthology.orchestra.run.smithi049.stdout:1804 write 0x71f3688 thru 0x72011ed (0xdb66 bytes) 2024-04-24T18:08:04.779 INFO:teuthology.orchestra.run.smithi049.stdout:1805 read 0x7f5f3cd thru 0x7f68900 (0x9534 bytes) 2024-04-24T18:08:04.780 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:04.780 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:04.780 INFO:teuthology.orchestra.run.smithi049.stdout:1806 punch from 0x22c288f to 0x22c66b5, (0x3e26 bytes) 2024-04-24T18:08:04.781 INFO:teuthology.orchestra.run.smithi049.stdout:1808 read 0x639f0cd thru 0x63a4db6 (0x5cea bytes) 2024-04-24T18:08:04.781 INFO:teuthology.orchestra.run.smithi049.stdout:1810 read 0x683bc00 thru 0x68463b5 (0xa7b6 bytes) 2024-04-24T18:08:04.781 INFO:teuthology.orchestra.run.smithi049.stdout:1811 write 0x3ef933b thru 0x3f03ddd (0xaaa3 bytes) 2024-04-24T18:08:04.791 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:04.792 INFO:teuthology.orchestra.run.smithi049.stdout:1812 write 0x86526da thru 0x865a7ff (0x8126 bytes) 2024-04-24T18:08:04.795 INFO:teuthology.orchestra.run.smithi049.stdout:1813 read 0x4c2fa66 thru 0x4c33843 (0x3dde bytes) 2024-04-24T18:08:04.798 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:04.805+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:04.801 INFO:teuthology.orchestra.run.smithi049.stdout:1814 writesame 0x9c65f87 thru 0x9c6c074 data_size 0x13(0x60ee bytes) 2024-04-24T18:08:04.811 INFO:teuthology.orchestra.run.smithi049.stdout:1816 writesame 0xd878e5d thru 0xd87b406 data_size 0x3(0x25aa bytes) 2024-04-24T18:08:04.823 INFO:teuthology.orchestra.run.smithi049.stdout:1817 trunc from 0xd87b407 to 0x28a9ea8 2024-04-24T18:08:04.834 INFO:teuthology.orchestra.run.smithi049.stdout:1819 write 0x2a8141 thru 0x2b0721 (0x85e1 bytes) 2024-04-24T18:08:04.847 INFO:teuthology.orchestra.run.smithi049.stdout:1820 write 0x83a375e thru 0x83ae3bb (0xac5e bytes) 2024-04-24T18:08:04.850 INFO:teuthology.orchestra.run.smithi049.stdout:1823 write 0x7485f7e thru 0x74936ec (0xd76f bytes) 2024-04-24T18:08:04.853 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.862 INFO:teuthology.orchestra.run.smithi049.stdout:1826 write 0xb079743 thru 0xb07a4f1 (0xdaf bytes) 2024-04-24T18:08:04.865 INFO:teuthology.orchestra.run.smithi049.stdout:1827 write 0x75c3e39 thru 0x75c9cd6 (0x5e9e bytes) 2024-04-24T18:08:04.867 INFO:teuthology.orchestra.run.smithi049.stdout:1829 read 0xa0e6763 thru 0xa0e8a81 (0x231f bytes) 2024-04-24T18:08:04.870 INFO:teuthology.orchestra.run.smithi049.stdout:1830 trunc from 0xb07a4f2 to 0x82dd641 2024-04-24T18:08:04.880 INFO:teuthology.orchestra.run.smithi049.stdout:1832 write 0x69f4078 thru 0x6a01b15 (0xda9e bytes) 2024-04-24T18:08:04.885 INFO:teuthology.orchestra.run.smithi049.stdout:1833 punch from 0x6053c59 to 0x6060bd9, (0xcf80 bytes) 2024-04-24T18:08:04.891 INFO:teuthology.orchestra.run.smithi049.stdout:1834 trunc from 0x82dd641 to 0xca1208a 2024-04-24T18:08:04.896 INFO:teuthology.orchestra.run.smithi049.stdout:1836 punch from 0x5631227 to 0x563d6ad, (0xc486 bytes) 2024-04-24T18:08:04.898 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:04.906+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:04.908 INFO:teuthology.orchestra.run.smithi049.stdout:1838 trunc from 0xca1208a to 0x20a132a 2024-04-24T18:08:04.920 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:04.920 INFO:teuthology.orchestra.run.smithi049.stdout:1841 read 0x8a16f3 thru 0x8ad970 (0xc27e bytes) 2024-04-24T18:08:04.926 INFO:teuthology.orchestra.run.smithi049.stdout:1844 writesame 0x33e2a59 thru 0x33e59ec data_size 0x15(0x2f94 bytes) 2024-04-24T18:08:04.936 INFO:teuthology.orchestra.run.smithi049.stdout:1845 trunc from 0x33e59ed to 0xbe9f094 2024-04-24T18:08:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:1846 punch from 0x113ba2b to 0x1140d8f, (0x5364 bytes) 2024-04-24T18:08:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:1848 read 0x2c6dfa thru 0x2cb882 (0x4a89 bytes) 2024-04-24T18:08:04.950 INFO:teuthology.orchestra.run.smithi049.stdout:1850 write 0xe3743ac thru 0xe3794a0 (0x50f5 bytes) 2024-04-24T18:08:04.959 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:04.959 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:04.964 INFO:teuthology.orchestra.run.smithi049.stdout:1851 trunc from 0xe3794a1 to 0x42f71ab 2024-04-24T18:08:04.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:04.973 INFO:teuthology.orchestra.run.smithi049.stdout:1852 read 0xb6f97d thru 0xb7402f (0x46b3 bytes) 2024-04-24T18:08:04.973 INFO:teuthology.orchestra.run.smithi049.stdout:1853 read 0x1ca0f54 thru 0x1caddfb (0xcea8 bytes) 2024-04-24T18:08:04.974 INFO:teuthology.orchestra.run.smithi049.stdout:1854 punch from 0x18974a8 to 0x18a0d86, (0x98de bytes) 2024-04-24T18:08:04.974 INFO:teuthology.orchestra.run.smithi049.stdout:1856 read 0xe1022d thru 0xe13db3 (0x3b87 bytes) 2024-04-24T18:08:04.990 INFO:teuthology.orchestra.run.smithi049.stdout:1857 write 0xca44f5a thru 0xca52f74 (0xe01b bytes) 2024-04-24T18:08:04.993 INFO:teuthology.orchestra.run.smithi049.stdout:1859 read 0x6c0e6f8 thru 0x6c13d2d (0x5636 bytes) 2024-04-24T18:08:04.993 INFO:teuthology.orchestra.run.smithi049.stdout:1860 writesame 0x59c617 thru 0x5a1224 data_size 0x21(0x4c0e bytes) 2024-04-24T18:08:04.999 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:05.007+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:05.006 INFO:teuthology.orchestra.run.smithi049.stdout:1864 punch from 0x91cc993 to 0x91d647f, (0x9aec bytes) 2024-04-24T18:08:05.006 INFO:teuthology.orchestra.run.smithi049.stdout:1865 write 0x1eae6c9 thru 0x1eb9b12 (0xb44a bytes) 2024-04-24T18:08:05.009 INFO:teuthology.orchestra.run.smithi049.stdout:1867 trunc from 0xca52f75 to 0xde588d0 2024-04-24T18:08:05.014 INFO:teuthology.orchestra.run.smithi049.stdout:1868 clone 15 order 20 su 262144 sc 10 2024-04-24T18:08:05.096 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:05.096 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:05.100 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:05.108+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:05.105 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:08:05.201 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:05.209+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:05.268 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:05.269 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:05.282 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:08:05.303 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:05.311+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:05.404 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:05.412+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:05.406 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:05.406 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:05.416 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:05.505 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:05.513+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:05.541 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:05.541 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:05.551 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:05.635 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:05.643+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:05.661 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:08:05.707 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:05.715+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:05.719 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone14 from 0xde588d0 (overlap 0x166ba61) to 0x0 2024-04-24T18:08:05.719 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:05.719 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:05.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:05.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:05.810 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #13, image image_client.0-clone13 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent14 2024-04-24T18:08:05.818 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:05.827+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:05.890 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:05.894 INFO:teuthology.orchestra.run.smithi049.stdout:1872 trunc from 0xde588d0 to 0xa9f381c 2024-04-24T18:08:05.905 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:05.905 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:05.907 INFO:teuthology.orchestra.run.smithi049.stdout:1873 read 0x7674144 thru 0x7676088 (0x1f45 bytes) 2024-04-24T18:08:05.907 INFO:teuthology.orchestra.run.smithi049.stdout:1874 read 0x79a8ddb thru 0x79b101c (0x8242 bytes) 2024-04-24T18:08:05.907 INFO:teuthology.orchestra.run.smithi049.stdout:1875 punch from 0x8dce2d7 to 0x8ddab8a, (0xc8b3 bytes) 2024-04-24T18:08:05.908 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:05.908 INFO:teuthology.orchestra.run.smithi049.stdout:1878 write 0x86d82e5 thru 0x86e7690 (0xf3ac bytes) 2024-04-24T18:08:05.908 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:05.916+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:05.911 INFO:teuthology.orchestra.run.smithi049.stdout:1879 read 0x5da961a thru 0x5db70c6 (0xdaad bytes) 2024-04-24T18:08:05.911 INFO:teuthology.orchestra.run.smithi049.stdout:1880 write 0x8a1d8fe thru 0x8a2be2d (0xe530 bytes) 2024-04-24T18:08:05.914 INFO:teuthology.orchestra.run.smithi049.stdout:1881 writesame 0x4a400a3 thru 0x4a4d94f data_size 0x1(0xd8ad bytes) 2024-04-24T18:08:05.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:05.947 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:05.948 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:05.959 INFO:teuthology.orchestra.run.smithi049.stdout:1883 trunc from 0xa9f381c to 0x3b5541 2024-04-24T18:08:05.987 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:06.009 INFO:teuthology.orchestra.run.smithi049.stdout:1884 writesame 0x50fb96e thru 0x510ad95 data_size 0x34(0xf428 bytes) 2024-04-24T18:08:06.009 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:06.017+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:06.012 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:06.026 INFO:teuthology.orchestra.run.smithi049.stdout:1885 write 0xea07eff thru 0xea0a0c1 (0x21c3 bytes) 2024-04-24T18:08:06.029 INFO:teuthology.orchestra.run.smithi049.stdout:1886 punch from 0x73ad398 to 0x73bc135, (0xed9d bytes) 2024-04-24T18:08:06.029 INFO:teuthology.orchestra.run.smithi049.stdout:1888 writesame 0xaf66ccb thru 0xaf67393 data_size 0x3(0x6c9 bytes) 2024-04-24T18:08:06.032 INFO:teuthology.orchestra.run.smithi049.stdout:1889 read 0x311400e thru 0x311b31e (0x7311 bytes) 2024-04-24T18:08:06.037 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:06.043 INFO:teuthology.orchestra.run.smithi049.stdout:1890 trunc from 0xea0a0c2 to 0x4d1dcdf 2024-04-24T18:08:06.055 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:06.059 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:06.064 INFO:teuthology.orchestra.run.smithi049.stdout:1892 write 0x6a6db3b thru 0x6a789dd (0xaea3 bytes) 2024-04-24T18:08:06.080 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:06.081 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:06.081 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:06.081 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:06.082 INFO:teuthology.orchestra.run.smithi049.stdout:1893 write 0xdfafccb thru 0xdfb2aeb (0x2e21 bytes) 2024-04-24T18:08:06.090 INFO:teuthology.orchestra.run.smithi049.stdout:1894 trunc from 0xdfb2aec to 0x8e70124 2024-04-24T18:08:06.098 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:06.101 INFO:teuthology.orchestra.run.smithi049.stdout:1895 punch from 0x7dfdee1 to 0x7e02a42, (0x4b61 bytes) 2024-04-24T18:08:06.103 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:06.107 INFO:teuthology.orchestra.run.smithi049.stdout:1896 write 0x995fe6b thru 0x99658b0 (0x5a46 bytes) 2024-04-24T18:08:06.109 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:06.117+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:06.111 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:08:06.111 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-24T18:08:06.112 INFO:teuthology.orchestra.run.smithi049.stdout:1897 punch from 0x17539b9 to 0x1756647, (0x2c8e bytes) 2024-04-24T18:08:06.112 INFO:teuthology.orchestra.run.smithi049.stdout:1898 read 0x8bf332c thru 0x8bf7ee8 (0x4bbd bytes) 2024-04-24T18:08:06.112 INFO:teuthology.orchestra.run.smithi049.stdout:1900 read 0x464fa66 thru 0x4653d06 (0x42a1 bytes) 2024-04-24T18:08:06.124 INFO:teuthology.orchestra.run.smithi049.stdout:1903 write 0xeaa0168 thru 0xeaaff8d (0xfe26 bytes) 2024-04-24T18:08:06.127 INFO:teuthology.orchestra.run.smithi049.stdout:1905 write 0x3f98888 thru 0x3fa8720 (0xfe99 bytes) 2024-04-24T18:08:06.128 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:06.130 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:06.132 INFO:teuthology.orchestra.run.smithi049.stdout:1906 read 0xe79ce53 thru 0xe7aaa19 (0xdbc7 bytes) 2024-04-24T18:08:06.132 INFO:teuthology.orchestra.run.smithi049.stdout:1909 read 0x5407d65 thru 0x5408bcf (0xe6b bytes) 2024-04-24T18:08:06.132 INFO:teuthology.orchestra.run.smithi049.stdout:1911 write 0x3600cd8 thru 0x36014f2 (0x81b bytes) 2024-04-24T18:08:06.144 INFO:teuthology.orchestra.run.smithi049.stdout:1912 trunc from 0xeaaff8e to 0x73534b2 2024-04-24T18:08:06.156 INFO:teuthology.orchestra.run.smithi049.stdout:1916 read 0x6a8b40 thru 0x6a8e98 (0x359 bytes) 2024-04-24T18:08:06.160 INFO:teuthology.orchestra.run.smithi049.stdout:1917 trunc from 0x73534b2 to 0x44d6173 2024-04-24T18:08:06.170 INFO:teuthology.orchestra.run.smithi049.stdout:1919 write 0x35bfa3 thru 0x361af8 (0x5b56 bytes) 2024-04-24T18:08:06.175 INFO:teuthology.orchestra.run.smithi049.stdout:1920 trunc from 0x44d6173 to 0x6c8210e 2024-04-24T18:08:06.181 INFO:teuthology.orchestra.run.smithi049.stdout:1921 trunc from 0x6c8210e to 0x78deff4 2024-04-24T18:08:06.187 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:06.197 INFO:teuthology.orchestra.run.smithi049.stdout:1923 write 0xa762496 thru 0xa768441 (0x5fac bytes) 2024-04-24T18:08:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:1924 read 0xa3d7473 thru 0xa3dd41b (0x5fa9 bytes) 2024-04-24T18:08:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:1925 punch from 0x5e75e39 to 0x5e79706, (0x38cd bytes) 2024-04-24T18:08:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:1926 read 0x3539c67 thru 0x354410d (0xa4a7 bytes) 2024-04-24T18:08:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:1929 read 0x7caa078 thru 0x7cb3b83 (0x9b0c bytes) 2024-04-24T18:08:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:1931 punch from 0x63d006 to 0x63e074, (0x106e bytes) 2024-04-24T18:08:06.201 INFO:teuthology.orchestra.run.smithi049.stdout:1932 write 0x75b4eb thru 0x768f50 (0xda66 bytes) 2024-04-24T18:08:06.202 INFO:teuthology.orchestra.run.smithi049.stdout:1933 read 0x71514e1 thru 0x71515fe (0x11e bytes) 2024-04-24T18:08:06.202 INFO:teuthology.orchestra.run.smithi049.stdout:1934 read 0x4beee1d thru 0x4bfc59e (0xd782 bytes) 2024-04-24T18:08:06.202 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:06.210 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:06.219+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:06.211 INFO:teuthology.orchestra.run.smithi049.stdout:1936 write 0xd848aa0 thru 0xd85218b (0x96ec bytes) 2024-04-24T18:08:06.215 INFO:teuthology.orchestra.run.smithi049.stdout:1937 read 0x8051f2a thru 0x80619d3 (0xfaaa bytes) 2024-04-24T18:08:06.215 INFO:teuthology.orchestra.run.smithi049.stdout:1938 write 0x7265fd4 thru 0x72702ce (0xa2fb bytes) 2024-04-24T18:08:06.218 INFO:teuthology.orchestra.run.smithi049.stdout:1939 read 0x52b206f thru 0x52b94f0 (0x7482 bytes) 2024-04-24T18:08:06.218 INFO:teuthology.orchestra.run.smithi049.stdout:1940 read 0x5e2f148 thru 0x5e35c0f (0x6ac8 bytes) 2024-04-24T18:08:06.233 INFO:teuthology.orchestra.run.smithi049.stdout:1944 trunc from 0xd85218c to 0x5a898 2024-04-24T18:08:06.260 INFO:teuthology.orchestra.run.smithi049.stdout:1946 read 0xc5ef thru 0xdf4b (0x195d bytes) 2024-04-24T18:08:06.260 INFO:teuthology.orchestra.run.smithi049.stdout:1947 read 0x386e9 thru 0x4619b (0xdab3 bytes) 2024-04-24T18:08:06.267 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:06.268 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:06.268 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:06.268 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:06.268 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:06.285 INFO:teuthology.orchestra.run.smithi049.stdout:1948 write 0xec3525a thru 0xec41ff0 (0xcd97 bytes) 2024-04-24T18:08:06.287 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:06.290 INFO:teuthology.orchestra.run.smithi049.stdout:1949 read 0x6974779 thru 0x697e270 (0x9af8 bytes) 2024-04-24T18:08:06.290 INFO:teuthology.orchestra.run.smithi049.stdout:1950 write 0x42dd93c thru 0x42e7d61 (0xa426 bytes) 2024-04-24T18:08:06.293 INFO:teuthology.orchestra.run.smithi049.stdout:1951 write 0x3a13610 thru 0x3a20ca9 (0xd69a bytes) 2024-04-24T18:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:1952 read 0x5b5aaa4 thru 0x5b61c7b (0x71d8 bytes) 2024-04-24T18:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:1953 punch from 0x71ab9e to 0x720a11, (0x5e73 bytes) 2024-04-24T18:08:06.297 INFO:teuthology.orchestra.run.smithi049.stdout:1955 clone 16 order 20 su 4096 sc 5 2024-04-24T18:08:06.311 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:06.319+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:06.413 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:06.420+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31810 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 2920448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.737173105, 2024-04-24T18:08:06.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052993206000000001, 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31810 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691739+0000", 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691742+0000", 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2800000000000002e-06 2024-04-24T18:08:06.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691747+0000", 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4319999999999998e-06 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691750+0000", 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.247e-06 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744520+0000", 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052769498999999997 2024-04-24T18:08:06.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744623+0000", 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000102808 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744733+0000", 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110282 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.723588345, 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:06.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:08:06.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.433 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32297 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3233792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.711214262, 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052981070999999998, 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32297 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.434 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717698+0000", 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717705+0000", 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5529999999999998e-06 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717711+0000", 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3849999999999999e-06 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717715+0000", 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770506+0000", 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052791267000000003 2024-04-24T18:08:06.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770587+0000", 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0783000000000002e-05 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770680+0000", 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2497000000000006e-05 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32299 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.711193259, 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053161606, 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32299 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717719+0000", 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717722+0000", 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.43e-06 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717729+0000", 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.576e-06 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717733+0000", 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0119999999999999e-06 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770707+0000", 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052974083999999998 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770791+0000", 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3418000000000007e-05 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770881+0000", 2024-04-24T18:08:06.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0490999999999998e-05 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32301 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.711175388999999, 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053336189999999999, 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32301 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:06.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717737+0000", 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717740+0000", 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4090000000000001e-06 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717746+0000", 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9989999999999997e-06 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717752+0000", 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0349999999999998e-06 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770915+0000", 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053162938 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770989+0000", 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4567999999999995e-05 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771074+0000", 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4598000000000006e-05 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32303 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3270656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.711062310999999, 2024-04-24T18:08:06.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053426976000000001, 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32303 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717850+0000", 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717854+0000", 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0509999999999998e-06 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717863+0000", 2024-04-24T18:08:06.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.346e-06 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717877+0000", 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3559e-05 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771102+0000", 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053224869000000001 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771184+0000", 2024-04-24T18:08:06.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2564000000000004e-05 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771278+0000", 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3178999999999997e-05 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32305 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3278848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:06.451 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.711030143, 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053587411000000001, 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32305 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717882+0000", 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717885+0000", 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8679999999999999e-06 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717892+0000", 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717896+0000", 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9360000000000001e-06 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771305+0000", 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053409310000000002 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771373+0000", 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8417999999999994e-05 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771470+0000", 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6827e-05 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32307 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3289088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:06.455 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.711013102000001, 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053767057, 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32307 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717899+0000", 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717902+0000", 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5019999999999999e-06 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717908+0000", 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7699999999999998e-06 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717911+0000", 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4979999999999998e-06 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771498+0000", 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053586747999999997 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771571+0000", 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3168000000000001e-05 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771667+0000", 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5810000000000006e-05 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32311 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3307520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.709716716999999, 2024-04-24T18:08:06.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053356705999999997, 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32311 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719199+0000", 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.88e-06 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719205+0000", 2024-04-24T18:08:06.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719209+0000", 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0749999999999999e-06 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772008+0000", 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052798572000000002 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772048+0000", 2024-04-24T18:08:06.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0238999999999999e-05 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772553+0000", 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050474099999999996 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32313 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3315712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.709698659000001, 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053105309000000003, 2024-04-24T18:08:06.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32313 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:06.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719216+0000", 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719223+0000", 2024-04-24T18:08:06.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3010000000000003e-06 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719226+0000", 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7610000000000001e-06 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772130+0000", 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052903238999999998 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772209+0000", 2024-04-24T18:08:06.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9486000000000004e-05 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772320+0000", 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011052099999999999 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32315 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3334144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.709682016, 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053250991999999997, 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.467 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32315 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:06.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719233+0000", 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1789999999999998e-06 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719240+0000", 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5689999999999999e-06 2024-04-24T18:08:06.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719244+0000", 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5870000000000001e-06 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772345+0000", 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053100972000000003 2024-04-24T18:08:06.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772413+0000", 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8071000000000006e-05 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772482+0000", 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8950999999999998e-05 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.471 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32317 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.709599955, 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053370316000000001, 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32317 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719312+0000", 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719316+0000", 2024-04-24T18:08:06.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9400000000000004e-06 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719326+0000", 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2439999999999999e-06 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719331+0000", 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7049999999999996e-06 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772583+0000", 2024-04-24T18:08:06.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053251860999999998 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772628+0000", 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5289e-05 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772683+0000", 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4882000000000001e-05 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.475 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32319 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3352576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.709576611999999, 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053459060000000003, 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32319 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:06.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719338+0000", 2024-04-24T18:08:06.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5490000000000001e-06 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719346+0000", 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5059999999999998e-06 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719350+0000", 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1999999999999996e-06 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772709+0000", 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053359182999999998 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772748+0000", 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8654e-05 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772795+0000", 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7437e-05 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32321 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3360768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.709558402000001, 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053627222000000002, 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32321 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719354+0000", 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719361+0000", 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7449999999999998e-06 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719367+0000", 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3940000000000001e-06 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719378+0000", 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1204e-05 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772830+0000", 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053451920999999999 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772896+0000", 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5623000000000005e-05 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772982+0000", 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5729000000000005e-05 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32327 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3389440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.708542194, 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053038750000000003, 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32327 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720370+0000", 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720373+0000", 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5830000000000001e-06 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720380+0000", 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1550000000000001e-06 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720384+0000", 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0260000000000001e-06 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773322+0000", 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052937495000000001 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773364+0000", 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2561999999999997e-05 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773409+0000", 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5333000000000001e-05 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32329 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3397632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.708524495000001, 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053169842000000002, 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32329 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720391+0000", 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7779999999999999e-06 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720396+0000", 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6729999999999997e-06 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720400+0000", 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7179999999999998e-06 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773435+0000", 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053035169 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773497+0000", 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1715000000000001e-05 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773558+0000", 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1171000000000001e-05 2024-04-24T18:08:06.491 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32331 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3416064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.708508366, 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053284320000000003, 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32331 2024-04-24T18:08:06.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720404+0000", 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720406+0000", 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3190000000000002e-06 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720412+0000", 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8359999999999997e-06 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720416+0000", 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000004e-06 2024-04-24T18:08:06.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773583+0000", 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053167076000000001 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773638+0000", 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105999999999997e-05 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773689+0000", 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0521e-05 2024-04-24T18:08:06.495 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32333 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3424256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.708490754, 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053395479000000003, 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32333 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.496 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720424+0000", 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720430+0000", 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720434+0000", 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8210000000000003e-06 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773721+0000", 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053287297999999997 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773762+0000", 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1245000000000002e-05 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773818+0000", 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5585999999999999e-05 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.499 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32388 4.3 4:c61fab1f:::rbd_object_map.117432c990a0.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.706388524999999, 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.055543581000000002, 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:06.500 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32388 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:06.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722527+0000", 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0319999999999999e-06 2024-04-24T18:08:06.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722535+0000", 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9170000000000006e-06 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722539+0000", 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9709999999999998e-06 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774649+0000", 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052110109000000002 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774719+0000", 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9562999999999998e-05 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775050+0000", 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000331078 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777149+0000", 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020996639999999998 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777996+0000", 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084650899999999996 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778027+0000", 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0818999999999997e-05 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778068+0000", 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1338000000000003e-05 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.8111135390000008, 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:06.510 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:06.515 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:06.517 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:06.518 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:06.525+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:06.617 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:06.626+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:06.653 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.0 to 1 (10000) 2024-04-24T18:08:06.653 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:08:06.669 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-24T18:08:06.694 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:06.695 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:06.695 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:06.695 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:06.711 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:06.718 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:06.726+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:06.727 INFO:teuthology.orchestra.run.smithi049.stdout:flattening image image_client.0-clone15 2024-04-24T18:08:06.819 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:06.827+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:06.869 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:06.869 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:06.869 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:06.869 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:06.881 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:06.885 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #14, image image_client.0-clone14 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent15 2024-04-24T18:08:06.887 INFO:teuthology.orchestra.run.smithi049.stdout:1956 write 0x854db97 thru 0x855bfc8 (0xe432 bytes) 2024-04-24T18:08:06.898 INFO:teuthology.orchestra.run.smithi049.stdout:1957 write 0x889d01f thru 0x88a5156 (0x8138 bytes) 2024-04-24T18:08:06.906 INFO:teuthology.orchestra.run.smithi049.stdout:1960 write 0x7f7585 thru 0x80630e (0xed8a bytes) 2024-04-24T18:08:06.920 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:06.928+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:06.932 INFO:teuthology.orchestra.run.smithi049.stdout:1961 read 0x7cad521 thru 0x7cb3fd2 (0x6ab2 bytes) 2024-04-24T18:08:06.932 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:06.944 INFO:teuthology.orchestra.run.smithi049.stdout:1964 trunc from 0xec41ff1 to 0x5031131 2024-04-24T18:08:06.974 INFO:teuthology.orchestra.run.smithi049.stdout:1965 trunc from 0x5031131 to 0xb3d3130 2024-04-24T18:08:06.980 INFO:teuthology.orchestra.run.smithi049.stdout:1966 read 0x63f1067 thru 0x63fd5f5 (0xc58f bytes) 2024-04-24T18:08:06.980 INFO:teuthology.orchestra.run.smithi049.stdout:1967 read 0x26cccd2 thru 0x26da09b (0xd3ca bytes) 2024-04-24T18:08:06.980 INFO:teuthology.orchestra.run.smithi049.stdout:1969 read 0x4ccd202 thru 0x4cd76f8 (0xa4f7 bytes) 2024-04-24T18:08:06.986 INFO:teuthology.orchestra.run.smithi049.stdout:1970 write 0xb9b7745 thru 0xb9bd0ea (0x59a6 bytes) 2024-04-24T18:08:06.996 INFO:teuthology.orchestra.run.smithi049.stdout:1971 read 0x798e1c8 thru 0x799d11e (0xef57 bytes) 2024-04-24T18:08:06.996 INFO:teuthology.orchestra.run.smithi049.stdout:1973 writesame 0x197a737 thru 0x1980d1d data_size 0x1(0x65e7 bytes) 2024-04-24T18:08:07.010 INFO:teuthology.orchestra.run.smithi049.stdout:1975 read 0x826fddd thru 0x827ab1e (0xad42 bytes) 2024-04-24T18:08:07.011 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:07.011 INFO:teuthology.orchestra.run.smithi049.stdout:1978 writesame 0x8092e9a thru 0x809b709 data_size 0x10(0x8870 bytes) 2024-04-24T18:08:07.021 INFO:teuthology.orchestra.run.smithi049.stdout:1980 punch from 0x814f8b3 to 0x815cda0, (0xd4ed bytes) 2024-04-24T18:08:07.021 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:07.029+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:07.022 INFO:teuthology.orchestra.run.smithi049.stdout:1981 trunc from 0xb9bd0eb to 0xc9b8993 2024-04-24T18:08:07.027 INFO:teuthology.orchestra.run.smithi049.stdout:1982 read 0x3cae19b thru 0x3cbaa0e (0xc874 bytes) 2024-04-24T18:08:07.035 INFO:teuthology.orchestra.run.smithi049.stdout:1984 trunc from 0xc9b8993 to 0x6bde54d 2024-04-24T18:08:07.047 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:07.047 INFO:teuthology.orchestra.run.smithi049.stdout:1987 read 0x429dcfe thru 0x42a008c (0x238f bytes) 2024-04-24T18:08:07.047 INFO:teuthology.orchestra.run.smithi049.stdout:1988 punch from 0x4b0853 to 0x4b7f35, (0x76e2 bytes) 2024-04-24T18:08:07.051 INFO:teuthology.orchestra.run.smithi049.stdout:1990 trunc from 0x6bde54d to 0x397e1c6 2024-04-24T18:08:07.067 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:07.067 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:07.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:07.067 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:07.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.067 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40940 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.067 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3666697859999992, 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036446457000000002, 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40940 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:07.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707548+0000", 2024-04-24T18:08:07.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.251e-06 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707551+0000", 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0019999999999998e-06 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707554+0000", 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743860+0000", 2024-04-24T18:08:07.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036306756000000003 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743915+0000", 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4423999999999999e-05 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743994+0000", 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9207000000000002e-05 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40941 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1490944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3666362349999996, 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036634427999999997, 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40941 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707583+0000", 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2089999999999999e-06 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707587+0000", 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1670000000000001e-06 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707590+0000", 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.554e-06 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744033+0000", 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036443542000000002 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744112+0000", 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8578999999999994e-05 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744215+0000", 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103607 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40942 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1523712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3666251890000005, 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036756906999999998, 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40942 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:07.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707594+0000", 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7090000000000001e-06 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707596+0000", 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707598+0000", 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9149999999999999e-06 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744245+0000", 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036646570000000003 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744295+0000", 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0343000000000002e-05 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744349+0000", 2024-04-24T18:08:07.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3661e-05 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40943 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3666165100000001, 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036956462000000002, 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.080 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40943 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707600+0000", 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707602+0000", 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.215e-06 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707605+0000", 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0429999999999999e-06 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707607+0000", 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-06 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744380+0000", 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036773477999999998 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744464+0000", 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4048000000000006e-05 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744557+0000", 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3043999999999999e-05 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40944 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1589248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3666085100000007, 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037054089999999998, 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.084 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40944 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707610+0000", 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1319999999999999e-06 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707613+0000", 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707614+0000", 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8819999999999999e-06 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744569+0000", 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036954539000000002 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744613+0000", 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3671999999999998e-05 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744663+0000", 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0145000000000002e-05 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40945 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3666006100000008, 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037248315999999997, 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.088 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40945 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707616+0000", 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707620+0000", 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1999999999999999e-06 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707623+0000", 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1669999999999999e-06 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707625+0000", 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1110000000000002e-06 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744693+0000", 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037068564999999998 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744773+0000", 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9172999999999996e-05 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744865+0000", 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2305000000000005e-05 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40946 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665900670000006, 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037340299, 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.092 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40946 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707628+0000", 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2699999999999999e-06 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707631+0000", 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9519999999999999e-06 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707633+0000", 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8619999999999999e-06 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744876+0000", 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037242550999999999 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744918+0000", 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2546000000000002e-05 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744967+0000", 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9320999999999998e-05 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40947 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665820110000002, 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037537860999999999, 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40947 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707636+0000", 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.186e-06 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707639+0000", 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707641+0000", 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.86e-06 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744997+0000", 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037356364000000003 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745076+0000", 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8355000000000005e-05 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745173+0000", 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7379000000000004e-05 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40948 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1720320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.366574087, 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037655182000000002, 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.100 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40948 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707644+0000", 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.269e-06 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707647+0000", 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9359999999999999e-06 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707649+0000", 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.815e-06 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745200+0000", 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037551189999999998 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745246+0000", 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6214999999999997e-05 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745298+0000", 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1960999999999999e-05 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40949 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665660580000001, 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037846454000000002, 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40949 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707652+0000", 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.13e-06 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707655+0000", 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707658+0000", 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5770000000000001e-06 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745329+0000", 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037671351999999998 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745408+0000", 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745498+0000", 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0043000000000005e-05 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40950 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665573779999995, 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037946947000000002, 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40950 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707661+0000", 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.15e-06 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707664+0000", 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.965e-06 2024-04-24T18:08:07.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707666+0000", 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745508+0000", 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037841977999999998 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745555+0000", 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7098999999999997e-05 2024-04-24T18:08:07.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745607+0000", 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1591999999999997e-05 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40951 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665488979999996, 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038137931999999999, 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40951 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707670+0000", 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7010000000000001e-06 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707673+0000", 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9119999999999998e-06 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707675+0000", 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9429999999999999e-06 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745637+0000", 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037962379999999997 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745715+0000", 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7488000000000001e-05 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745806+0000", 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1723000000000003e-05 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40952 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665402719999999, 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038236762000000001, 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40952 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707678+0000", 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.093e-06 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707681+0000", 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707683+0000", 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.951e-06 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745817+0000", 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038133854000000002 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745862+0000", 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5027000000000001e-05 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745914+0000", 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2120999999999998e-05 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40953 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.122 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665322389999996, 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038424458000000002, 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40953 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707686+0000", 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.567e-06 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707689+0000", 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9220000000000001e-06 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707691+0000", 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9369999999999998e-06 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745943+0000", 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038252047999999997 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746018+0000", 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4938000000000007e-05 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746109+0000", 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1255000000000006e-05 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40954 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1916928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:07.126 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665237300000008, 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038528139000000003, 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40954 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707693+0000", 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.065e-06 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707697+0000", 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.881e-06 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707699+0000", 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.88e-06 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746120+0000", 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038420937000000002 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746171+0000", 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0555e-05 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746222+0000", 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1025e-05 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40955 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665159100000004, 2024-04-24T18:08:07.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718855000000003, 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40955 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707702+0000", 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.212e-06 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707706+0000", 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.534e-06 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707708+0000", 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746254+0000", 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038545870000000003 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746331+0000", 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7805000000000005e-05 2024-04-24T18:08:07.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746420+0000", 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8721000000000007e-05 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40956 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3665072919999997, 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038823347000000001, 2024-04-24T18:08:07.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40956 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:07.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707711+0000", 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1659999999999999e-06 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707714+0000", 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.937e-06 2024-04-24T18:08:07.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707716+0000", 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9920000000000002e-06 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746435+0000", 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718869000000003 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746482+0000", 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6882999999999998e-05 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746533+0000", 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1711e-05 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40957 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:07.138 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3664990180000007, 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039015052000000001, 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40957 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707721+0000", 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3110000000000001e-06 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707725+0000", 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2669999999999998e-06 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707727+0000", 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746563+0000", 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038836751000000003 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746641+0000", 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7900999999999998e-05 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746733+0000", 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2015999999999994e-05 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40958 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2048000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3664883250000006, 2024-04-24T18:08:07.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039110399999999997, 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40958 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707730+0000", 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2640000000000001e-06 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707733+0000", 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9170000000000002e-06 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707735+0000", 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746744+0000", 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039008958000000003 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746789+0000", 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5019999999999999e-05 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746839+0000", 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0531999999999997e-05 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40959 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.146 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.3664801519999994, 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039290798000000002, 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40959 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707738+0000", 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.24e-06 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707742+0000", 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6200000000000001e-06 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707744+0000", 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.615e-06 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746859+0000", 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039115095000000003 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746938+0000", 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.747028+0000", 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9742999999999998e-05 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.150 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:07.152 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:07.152 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:07.153 INFO:teuthology.orchestra.run.smithi049.stdout:1992 write 0x2967ff0 thru 0x2972888 (0xa899 bytes) 2024-04-24T18:08:07.153 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:07.153 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:07.153 INFO:teuthology.orchestra.run.smithi049.stdout:1995 clone 17 order 24 su 4194304 sc 9 2024-04-24T18:08:07.153 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:07.159+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:07.253 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:07.261+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:07.322 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:07.322 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:07.322 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:07.322 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:07.332 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:07.373 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:07.381+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:07.455 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:07.463+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:07.501 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:07.501 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:07.501 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:07.501 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:07.501 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:07.511 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:07.595 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:07.603+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:07.657 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:07.664+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:07.660 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.2 to 1 (10000) 2024-04-24T18:08:07.673 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:6850 4.1c 4:3d3e8d61:::rbd_data.11749ffb1524.000000000000000a:head [stat,write-same] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e37)", 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:48.118841+0000", 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 19.577538501999999, 2024-04-24T18:08:07.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066472299999999996, 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 6850 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118841+0000", 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118841+0000", 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118839+0000", 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118848+0000", 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1929999999999999e-06 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118867+0000", 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8147000000000001e-05 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118875+0000", 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9840000000000001e-06 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118915+0000", 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0797999999999998e-05 2024-04-24T18:08:07.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.118974+0000", 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8913999999999999e-05 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.120364+0000", 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001389794 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125153+0000", 2024-04-24T18:08:07.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0047890989999999998 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125430+0000", 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00027735900000000003 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125462+0000", 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1741e-05 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:48.125488+0000", 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6044e-05 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33424 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.779002+0000", 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.917377328000001, 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065180560000000004, 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33424 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779002+0000", 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779002+0000", 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778999+0000", 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999971 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779245+0000", 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00024558499999999999 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779267+0000", 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1894000000000001e-05 2024-04-24T18:08:07.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779275+0000", 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3210000000000005e-06 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779336+0000", 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0927000000000001e-05 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779427+0000", 2024-04-24T18:08:07.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1032000000000005e-05 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779666+0000", 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000238874 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.782980+0000", 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0033137829999999998 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785422+0000", 2024-04-24T18:08:07.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0024418399999999998 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785482+0000", 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9886999999999998e-05 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785520+0000", 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8686000000000002e-05 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33425 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:head [delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.779290+0000", 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.917089711999999, 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006444094, 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33425 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779290+0000", 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779290+0000", 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779289+0000", 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779293+0000", 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8879999999999999e-06 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779306+0000", 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3115999999999999e-05 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.779311+0000", 2024-04-24T18:08:07.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4719999999999997e-06 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.780818+0000", 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001506949 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.780904+0000", 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6007999999999994e-05 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.781092+0000", 2024-04-24T18:08:07.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018842300000000001 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.783464+0000", 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023717180000000001 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785650+0000", 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002186571 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785695+0000", 2024-04-24T18:08:07.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4993000000000001e-05 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.785734+0000", 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8615999999999998e-05 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35123 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 196608~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058125+0000", 2024-04-24T18:08:07.707 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638254766999999, 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065029679999999996, 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35123 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058125+0000", 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058125+0000", 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058124+0000", 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058127+0000", 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1320000000000001e-06 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058132+0000", 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7350000000000001e-06 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058136+0000", 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4929999999999998e-06 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.063278+0000", 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051423079999999999 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.063355+0000", 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6891000000000006e-05 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064628+0000", 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0012727089999999999 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35124 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 286720~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.711 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638239628000001, 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066858179999999996, 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35124 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058140+0000", 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058142+0000", 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1689999999999999e-06 2024-04-24T18:08:07.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058147+0000", 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6659999999999996e-06 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058151+0000", 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064658+0000", 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065067579999999996 2024-04-24T18:08:07.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064740+0000", 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2124000000000002e-05 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064826+0000", 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5892999999999999e-05 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35125 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 378880~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058161+0000", 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638218821000001, 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.006851673, 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35125 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058161+0000", 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058161+0000", 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058160+0000", 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058163+0000", 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4789999999999999e-06 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058169+0000", 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4489999999999996e-06 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058173+0000", 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1559999999999997e-06 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064853+0000", 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0066795270000000002 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.064929+0000", 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6259000000000003e-05 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065012+0000", 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3146999999999995e-05 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35126 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 471040~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058178+0000", 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638201768, 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0070301249999999999, 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35126 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058178+0000", 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058178+0000", 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058177+0000", 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058180+0000", 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.339e-06 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058185+0000", 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2950000000000001e-06 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058189+0000", 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.455e-06 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065040+0000", 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0068511880000000002 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065117+0000", 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6890000000000004e-05 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065208+0000", 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1278000000000003e-05 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35266 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 18432~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638186607, 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0072003559999999998, 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35266 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058193+0000", 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058195+0000", 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.334e-06 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058201+0000", 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.37e-06 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058205+0000", 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5619999999999999e-06 2024-04-24T18:08:07.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065235+0000", 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0070298649999999997 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065310+0000", 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5741000000000005e-05 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065393+0000", 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2822000000000001e-05 2024-04-24T18:08:07.727 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35267 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 110592~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638170119, 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0073675850000000003, 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35267 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.728 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058209+0000", 2024-04-24T18:08:07.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058212+0000", 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4590000000000001e-06 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058217+0000", 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9669999999999998e-06 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058221+0000", 2024-04-24T18:08:07.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.811e-06 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065419+0000", 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071978709999999998 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065494+0000", 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5224999999999999e-05 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065577+0000", 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2609999999999997e-05 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35268 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 202752~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058226+0000", 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638153831, 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0075397320000000004, 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.732 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35268 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058226+0000", 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058226+0000", 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058225+0000", 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058228+0000", 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058233+0000", 2024-04-24T18:08:07.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6550000000000001e-06 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058237+0000", 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6950000000000002e-06 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065604+0000", 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007366877 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065683+0000", 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9515e-05 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065765+0000", 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2191e-05 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35269 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 294912~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638138113, 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0077089660000000003, 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35269 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058241+0000", 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058243+0000", 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3e-06 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058249+0000", 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8320000000000002e-06 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058253+0000", 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8759999999999998e-06 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065792+0000", 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0075390250000000004 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065868+0000", 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5660000000000004e-05 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065950+0000", 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2601000000000004e-05 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.743 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35270 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 385024~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638121649, 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0078762390000000002, 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35270 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058258+0000", 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058260+0000", 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4080000000000001e-06 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058266+0000", 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8889999999999999e-06 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058270+0000", 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7220000000000001e-06 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.065977+0000", 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0077071939999999997 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066052+0000", 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5035000000000002e-05 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066134+0000", 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2338999999999999e-05 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.747 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35271 4.1c 4:39f17d49:::rbd_data.117432c990a0.0000000000000013:a [read 477184~6144] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.058274+0000", 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.638105787000001, 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0081649119999999999, 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35271 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.748 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058274+0000", 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058274+0000", 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058273+0000", 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058276+0000", 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0549999999999998e-06 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058281+0000", 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7239999999999997e-06 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.058285+0000", 2024-04-24T18:08:07.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5310000000000002e-06 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066237+0000", 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0079519829999999993 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066333+0000", 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6749999999999994e-05 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.066439+0000", 2024-04-24T18:08:07.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000105169 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37455 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.795782+0000", 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.900597202, 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053832250000000002, 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37455 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795782+0000", 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795782+0000", 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795781+0000", 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795789+0000", 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3159999999999999e-06 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795801+0000", 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2840000000000001e-05 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.795808+0000", 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0899999999999999e-06 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797600+0000", 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017916620000000001 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797703+0000", 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010255 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.798106+0000", 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00040297299999999999 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801065+0000", 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0029588520000000001 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801072+0000", 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3699999999999997e-06 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801120+0000", 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7842e-05 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801166+0000", 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5840000000000002e-05 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37458 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:07.757 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.899419353000001, 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.005658667, 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37458 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796960+0000", 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796964+0000", 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3640000000000001e-06 2024-04-24T18:08:07.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796972+0000", 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0369999999999995e-06 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.796977+0000", 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0629999999999996e-06 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.800493+0000", 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0035157029999999998 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.800539+0000", 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6363000000000003e-05 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.800749+0000", 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000209614 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802040+0000", 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001291726 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802530+0000", 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00048976499999999997 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802581+0000", 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0992999999999998e-05 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802619+0000", 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7693e-05 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.762 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37459 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.797109+0000", 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.899270684999999, 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0072136969999999998, 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37459 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.763 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797109+0000", 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797109+0000", 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797108+0000", 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797114+0000", 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0270000000000001e-06 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797125+0000", 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1119e-05 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.797134+0000", 2024-04-24T18:08:07.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4120000000000001e-06 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801288+0000", 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041545030000000004 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801362+0000", 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3532e-05 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:07.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.801660+0000", 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00029880400000000001 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.802752+0000", 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001091369 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.804257+0000", 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015050910000000001 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.804295+0000", 2024-04-24T18:08:07.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8139999999999997e-05 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.804323+0000", 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7572e-05 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_repop(client.4468.0:37548 4.19 e43/26 4:9d690b33:::rbd_data.1174b285a870.0000000000000001:head v 43'55, mlcod=42'53)", 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.843919+0000", 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.852460950999999, 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0090564110000000003, 2024-04-24T18:08:07.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.843919+0000", 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.843919+0000", 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.843917+0000", 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.844810+0000", 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00089266599999999997 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.844974+0000", 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000163859 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.844980+0000", 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9660000000000001e-06 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845069+0000", 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9844000000000002e-05 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845090+0000", 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0069e-05 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.852929+0000", 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0078399070000000001 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.852975+0000", 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5578999999999998e-05 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:37550 4.4 4:252d932e:::rbd_object_map.1174b285a870:head [call lock.assert_locked in=17b,call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e43)", 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.845638+0000", 2024-04-24T18:08:07.772 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.850741144000001, 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0079207649999999998, 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 37550 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845638+0000", 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845638+0000", 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845636+0000", 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999976 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845645+0000", 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1160000000000005e-06 2024-04-24T18:08:07.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845657+0000", 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1799e-05 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845664+0000", 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845832+0000", 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000168233 2024-04-24T18:08:07.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.845934+0000", 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101734 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 3", 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.846328+0000", 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00039441699999999999 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.847343+0000", 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010148819999999999 2024-04-24T18:08:07.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.853505+0000", 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061618580000000001 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.853537+0000", 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2351999999999998e-05 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.853559+0000", 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2112000000000001e-05 2024-04-24T18:08:07.777 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40434 4.1c 4:3ae45a05:::rbd_data.1174f9704cae.0000000000000006:head [stat,delete] snapc d=[d] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.696803+0000", 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.9995766899999996, 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0071709690000000001, 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40434 2024-04-24T18:08:07.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696803+0000", 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696803+0000", 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696801+0000", 2024-04-24T18:08:07.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696809+0000", 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0080000000000002e-06 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696831+0000", 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1225000000000001e-05 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696840+0000", 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6490000000000003e-06 2024-04-24T18:08:07.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696893+0000", 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2512999999999997e-05 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.696980+0000", 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6694999999999996e-05 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.697173+0000", 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00019387399999999999 2024-04-24T18:08:07.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.699556+0000", 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023828569999999999 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.703880+0000", 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043240190000000001 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.703933+0000", 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2735999999999998e-05 2024-04-24T18:08:07.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.703974+0000", 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0822999999999999e-05 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:42560 4.1c 4:3c3da860:::rbd_data.1174578192f5.0000000000000014:head [stat,truncate 262144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:58.250773+0000", 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.4456061420000008, 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0091032000000000005, 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:07.783 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 42560 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250773+0000", 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250773+0000", 2024-04-24T18:08:07.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250772+0000", 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250780+0000", 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2099999999999993e-06 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250798+0000", 2024-04-24T18:08:07.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.853e-05 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250806+0000", 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.712e-06 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250855+0000", 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8823000000000003e-05 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:07.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.250935+0000", 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0557999999999997e-05 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.251072+0000", 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136775 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.251688+0000", 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00061615400000000005 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.259778+0000", 2024-04-24T18:08:07.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0080894409999999993 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.259837+0000", 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8857999999999998e-05 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:58.259877+0000", 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.994e-05 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:07.788 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:07.789 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:07.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:07.791 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone16 from 0x397e1c6 (overlap 0x397e1c6) to 0x2dfe7d1 2024-04-24T18:08:07.791 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:07.798+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:07.807 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #15, image image_client.0-clone15 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent16 2024-04-24T18:08:07.891 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:07.900+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:07.964 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:07.964 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:07.965 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:07.965 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:07.976 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:07.992 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:08.001+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:08.074 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.093 INFO:teuthology.orchestra.run.smithi049.stdout:1997 write 0xb599e0b thru 0xb5a04f9 (0x66ef bytes) 2024-04-24T18:08:08.094 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:08.101+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:08.096 INFO:teuthology.orchestra.run.smithi049.stdout:1998 read 0x91334d0 thru 0x913e00d (0xab3e bytes) 2024-04-24T18:08:08.096 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.096 INFO:teuthology.orchestra.run.smithi049.stdout:2000 write 0x84120ab thru 0x841dab3 (0xba09 bytes) 2024-04-24T18:08:08.105 INFO:teuthology.orchestra.run.smithi049.stdout:2002 read 0xe7d365 thru 0xe83ae1 (0x677d bytes) 2024-04-24T18:08:08.105 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.105 INFO:teuthology.orchestra.run.smithi049.stdout:2005 punch from 0x395836f to 0x39632ef, (0xaf80 bytes) 2024-04-24T18:08:08.105 INFO:teuthology.orchestra.run.smithi049.stdout:2006 write 0x1d8e749 thru 0x1d90db8 (0x2670 bytes) 2024-04-24T18:08:08.110 INFO:teuthology.orchestra.run.smithi049.stdout:2007 read 0x9469c62 thru 0x9477ed6 (0xe275 bytes) 2024-04-24T18:08:08.110 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.110 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.110 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.110 INFO:teuthology.orchestra.run.smithi049.stdout:2016 read 0x8db5d6f thru 0x8db7658 (0x18ea bytes) 2024-04-24T18:08:08.114 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:08.114 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:08.114 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:08.115 INFO:teuthology.orchestra.run.smithi049.stdout:2018 trunc from 0xb5a04fa to 0x749b615 2024-04-24T18:08:08.127 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:08.151 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:08.166 INFO:teuthology.orchestra.run.smithi049.stdout:2020 read 0x3d762a6 thru 0x3d7a03f (0x3d9a bytes) 2024-04-24T18:08:08.166 INFO:teuthology.orchestra.run.smithi049.stdout:2022 write 0x8178fd thru 0x826c79 (0xf37d bytes) 2024-04-24T18:08:08.178 INFO:teuthology.orchestra.run.smithi049.stdout:2023 write 0x93944e6 thru 0x9395a10 (0x152b bytes) 2024-04-24T18:08:08.181 INFO:teuthology.orchestra.run.smithi049.stdout:2024 punch from 0x73ebe97 to 0x73fa5e7, (0xe750 bytes) 2024-04-24T18:08:08.181 INFO:teuthology.orchestra.run.smithi049.stdout:2025 write 0x6a3f3b0 thru 0x6a41d94 (0x29e5 bytes) 2024-04-24T18:08:08.183 INFO:teuthology.orchestra.run.smithi049.stdout:2029 write 0x4f56c37 thru 0x4f5b8ab (0x4c75 bytes) 2024-04-24T18:08:08.191 INFO:teuthology.orchestra.run.smithi049.stdout:2032 read 0x102400d thru 0x1024628 (0x61c bytes) 2024-04-24T18:08:08.193 INFO:teuthology.orchestra.run.smithi049.stdout:2033 trunc from 0x9395a11 to 0x70eb2dc 2024-04-24T18:08:08.194 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:08.202+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:08.209 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.209 INFO:teuthology.orchestra.run.smithi049.stdout:2035 read 0x18626f6 thru 0x1870643 (0xdf4e bytes) 2024-04-24T18:08:08.210 INFO:teuthology.orchestra.run.smithi049.stdout:2036 write 0x311a0de thru 0x311fb09 (0x5a2c bytes) 2024-04-24T18:08:08.212 INFO:teuthology.orchestra.run.smithi049.stdout:2037 clone 18 order 19 su 262144 sc 3 2024-04-24T18:08:08.295 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:08.303+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33192 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1382400~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.774898+0000", 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.538984762, 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047930773000000003, 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.308 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33192 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774898+0000", 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774898+0000", 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774897+0000", 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774905+0000", 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2009999999999999e-06 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774912+0000", 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4239999999999998e-06 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774916+0000", 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8249999999999998e-06 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.822653+0000", 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047737363999999997 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.822731+0000", 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7692000000000001e-05 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.822828+0000", 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7626000000000003e-05 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33193 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1433600~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.774920+0000", 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.538962160000001, 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048094625000000002, 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33193 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774920+0000", 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774920+0000", 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774920+0000", 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774922+0000", 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4169999999999999e-06 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774930+0000", 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6790000000000005e-06 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774934+0000", 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6339999999999999e-06 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.822859+0000", 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047925253000000001 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.822930+0000", 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0995999999999996e-05 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.823015+0000", 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5023000000000002e-05 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.316 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33194 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1474560~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.775085+0000", 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.538797408000001, 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048062050000000002, 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33194 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.317 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775085+0000", 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775085+0000", 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774937+0000", 2024-04-24T18:08:08.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9998522 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775089+0000", 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015170099999999999 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775097+0000", 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5480000000000002e-06 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775103+0000", 2024-04-24T18:08:08.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2490000000000001e-06 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.823047+0000", 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047944850999999997 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.823095+0000", 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7638e-05 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.823147+0000", 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1952e-05 2024-04-24T18:08:08.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33199 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1587200~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.776312+0000", 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.537570350999999, 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047754644999999998, 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33199 2024-04-24T18:08:08.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776312+0000", 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776312+0000", 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776312+0000", 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776314+0000", 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776320+0000", 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4110000000000001e-06 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776331+0000", 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0781000000000001e-05 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.823906+0000", 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047574370999999997 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.823976+0000", 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0310999999999998e-05 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.824067+0000", 2024-04-24T18:08:08.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0703000000000002e-05 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33200 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1628160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.776336+0000", 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.537546273, 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047878711999999997, 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33200 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776336+0000", 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776336+0000", 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776336+0000", 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776338+0000", 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5679999999999998e-06 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776344+0000", 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1489999999999999e-06 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776350+0000", 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.032e-06 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.824093+0000", 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047742442000000003 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.824153+0000", 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9716999999999998e-05 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.824215+0000", 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2209e-05 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.329 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33201 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1679360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.776355+0000", 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.537527318, 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048073115, 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33201 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.330 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776355+0000", 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776355+0000", 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776355+0000", 2024-04-24T18:08:08.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776357+0000", 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3910000000000001e-06 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776363+0000", 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1940000000000003e-06 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.776368+0000", 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2150000000000003e-06 2024-04-24T18:08:08.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.824254+0000", 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047886541999999997 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.824331+0000", 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6648000000000002e-05 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.824428+0000", 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7461999999999996e-05 2024-04-24T18:08:08.333 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33278 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1454080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.784135+0000", 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.529747656, 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047763927999999997, 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33278 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784135+0000", 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784135+0000", 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784135+0000", 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784136+0000", 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.31e-06 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784139+0000", 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.819e-06 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784141+0000", 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-06 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.831813+0000", 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047672175999999997 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.831849+0000", 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6099000000000001e-05 2024-04-24T18:08:08.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.831899+0000", 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9824000000000003e-05 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33279 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1495040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.784143+0000", 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.529739849, 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047983014999999997, 2024-04-24T18:08:08.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33279 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784143+0000", 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784143+0000", 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784142+0000", 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784143+0000", 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.144e-06 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784146+0000", 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8480000000000001e-06 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784148+0000", 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.801e-06 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.831933+0000", 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047784897 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832018+0000", 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5397999999999998e-05 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832126+0000", 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010713900000000001 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.342 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33280 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1505280~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.784150+0000", 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.52973223, 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048085127999999998, 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33280 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784150+0000", 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784150+0000", 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784150+0000", 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784151+0000", 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1370000000000001e-06 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784162+0000", 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.064e-05 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784166+0000", 2024-04-24T18:08:08.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832137+0000", 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047971860999999998 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832184+0000", 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6669000000000003e-05 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832235+0000", 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.117e-05 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33281 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1546240~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.784170+0000", 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.529712242, 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048292728, 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33281 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784170+0000", 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784170+0000", 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784170+0000", 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784172+0000", 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3709999999999998e-06 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784178+0000", 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784182+0000", 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7299999999999999e-06 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832272+0000", 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048090582999999999 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832356+0000", 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3873000000000001e-05 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832463+0000", 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000106567 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.351 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33282 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1597440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.784186+0000", 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.529696341999999, 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048374163999999997, 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33282 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784186+0000", 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784186+0000", 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784186+0000", 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784188+0000", 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2970000000000002e-06 2024-04-24T18:08:08.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784192+0000", 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3719999999999998e-06 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.784195+0000", 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832475+0000", 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048280971999999998 2024-04-24T18:08:08.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832512+0000", 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6366999999999999e-05 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.832560+0000", 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8343000000000001e-05 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33352 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1116160~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.790662+0000", 2024-04-24T18:08:08.355 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.523219868, 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047750844000000001, 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33352 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.790662+0000", 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.790662+0000", 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.790662+0000", 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.790664+0000", 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4760000000000001e-06 2024-04-24T18:08:08.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.790670+0000", 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.874e-06 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.790674+0000", 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6519999999999999e-06 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.838327+0000", 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047652996000000003 2024-04-24T18:08:08.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.838364+0000", 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7120999999999999e-05 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.838413+0000", 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9259999999999999e-05 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.359 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:33420 4.11 4:8dc01f57:::rbd_data.117432c990a0.000000000000001a:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 524288 write_size 524288,delete] snapc a=[a] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.798160+0000", 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.515722598, 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047777467999999997, 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 33420 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.798160+0000", 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.798160+0000", 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.796292+0000", 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9981327 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.798164+0000", 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001871618 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.798173+0000", 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5240000000000006e-06 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.798178+0000", 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5539999999999999e-06 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.843312+0000", 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.045134134999999999 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.843363+0000", 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0331999999999999e-05 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 7", 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.843499+0000", 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013604599999999999 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.845877+0000", 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023786889999999998 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:08.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.845883+0000", 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3739999999999997e-06 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.845914+0000", 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1090999999999998e-05 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.845937+0000", 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3533999999999999e-05 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.364 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35915 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 952320~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.135496+0000", 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.178386290000001, 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047822433999999997, 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35915 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135496+0000", 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135496+0000", 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135496+0000", 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135498+0000", 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135503+0000", 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4439999999999999e-06 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135508+0000", 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1069999999999998e-06 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183139+0000", 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047631469000000003 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183226+0000", 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6545000000000006e-05 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183319+0000", 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2929000000000004e-05 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35916 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 962560~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.135529+0000", 2024-04-24T18:08:08.368 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.178353067, 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048038386000000002, 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35916 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135529+0000", 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135529+0000", 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135529+0000", 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135532+0000", 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.01e-06 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135539+0000", 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9099999999999999e-06 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135543+0000", 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4499999999999997e-06 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183338+0000", 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047795031000000002 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183439+0000", 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100562 2024-04-24T18:08:08.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183568+0000", 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000128944 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35917 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 972800~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.135548+0000", 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.178334336000001, 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048231158000000003, 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.372 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35917 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135548+0000", 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135548+0000", 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135547+0000", 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135550+0000", 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6730000000000001e-06 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135556+0000", 2024-04-24T18:08:08.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8660000000000002e-06 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135560+0000", 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8700000000000002e-06 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183606+0000", 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048045728000000003 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183685+0000", 2024-04-24T18:08:08.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9605999999999996e-05 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183779+0000", 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4059000000000002e-05 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35918 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1433600~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.135564+0000", 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.178318119, 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048474537999999998, 2024-04-24T18:08:08.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35918 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135564+0000", 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135564+0000", 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135564+0000", 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135566+0000", 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.311e-06 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135573+0000", 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0870000000000001e-06 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135577+0000", 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183824+0000", 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048247142999999999 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.183922+0000", 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7844999999999996e-05 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184039+0000", 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116642 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35919 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1413120~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.380 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.135582+0000", 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.178300567999999, 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048671761000000001, 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35919 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135582+0000", 2024-04-24T18:08:08.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135582+0000", 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135581+0000", 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135584+0000", 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4600000000000002e-06 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135590+0000", 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0390000000000001e-06 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135594+0000", 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0600000000000001e-06 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184078+0000", 2024-04-24T18:08:08.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048483859999999997 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184165+0000", 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7083999999999995e-05 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184254+0000", 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8622999999999995e-05 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.384 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35920 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1423360~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.135598+0000", 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.178284238, 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048872572000000003, 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35920 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135598+0000", 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135598+0000", 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135598+0000", 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135600+0000", 2024-04-24T18:08:08.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.605e-06 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135606+0000", 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.135610+0000", 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7620000000000001e-06 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184294+0000", 2024-04-24T18:08:08.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048683653 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184376+0000", 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2068000000000001e-05 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184471+0000", 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4914000000000006e-05 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:35923 4.19 4:980b0ada:::rbd_data.1174bc8f97cd.0000000000000006:8 [read 1894400~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:53.137204+0000", 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 15.176678826, 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047752230999999999, 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 35923 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:08.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.137204+0000", 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.137204+0000", 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.137203+0000", 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:08.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.137205+0000", 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4270000000000001e-06 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.137208+0000", 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1999999999999999e-06 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.137210+0000", 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.148e-06 2024-04-24T18:08:08.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184860+0000", 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.047650344999999997 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184903+0000", 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2673000000000003e-05 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:53.184956+0000", 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2669e-05 2024-04-24T18:08:08.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.393 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.393 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.393 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:08.393 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:08.393 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:08.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:08.395 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:08.395 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:08.395 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:08.395 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:08.396 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:08.396 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:08.404+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:08.415 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:08.435 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:08.453 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:08.455 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:08.497 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:08.505+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:08.536 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:08.536 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:08.547 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:08.598 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:08.606+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:08.661 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.3 to 1 (10000) 2024-04-24T18:08:08.671 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:08:08.675 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-24T18:08:08.698 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:08.707+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:08.715 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:08.715 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:08.724 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone17 from 0x70eb2dc (overlap 0x397e1c6) to 0x227eddd 2024-04-24T18:08:08.726 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:08.800 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:08.808+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:08.873 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #16, image image_client.0-clone16 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent17 2024-04-24T18:08:08.898 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:08.898 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:08.901 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:08.909+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:08.909 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:08.956 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.956 INFO:teuthology.orchestra.run.smithi049.stdout:2042 punch from 0xe57ede to 0xe5f448, (0x756a bytes) 2024-04-24T18:08:08.960 INFO:teuthology.orchestra.run.smithi049.stdout:2043 punch from 0x4f16b96 to 0x4f1d724, (0x6b8e bytes) 2024-04-24T18:08:08.965 INFO:teuthology.orchestra.run.smithi049.stdout:2044 trunc from 0x70eb2dc to 0x29aeddb 2024-04-24T18:08:08.976 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.976 INFO:teuthology.orchestra.run.smithi049.stdout:2046 punch from 0x2616837 to 0x261d8f9, (0x70c2 bytes) 2024-04-24T18:08:08.977 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.989 INFO:teuthology.orchestra.run.smithi049.stdout:2048 write 0x95efb6c thru 0x95f0f75 (0x140a bytes) 2024-04-24T18:08:08.992 INFO:teuthology.orchestra.run.smithi049.stdout:2049 write 0x5c5f6dc thru 0x5c640f0 (0x4a15 bytes) 2024-04-24T18:08:08.995 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.995 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:08.995 INFO:teuthology.orchestra.run.smithi049.stdout:2053 writesame 0x2a490b2 thru 0x2a49ca2 data_size 0x1(0xbf1 bytes) 2024-04-24T18:08:09.001 INFO:teuthology.orchestra.run.smithi049.stdout:2055 punch from 0x45aa486 to 0x45b13a9, (0x6f23 bytes) 2024-04-24T18:08:09.001 INFO:teuthology.orchestra.run.smithi049.stdout:2057 read 0x2bc1346 thru 0x2bc7fd1 (0x6c8c bytes) 2024-04-24T18:08:09.001 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:09.009+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:09.005 INFO:teuthology.orchestra.run.smithi049.stdout:2058 trunc from 0x95f0f76 to 0xd59e249 2024-04-24T18:08:09.010 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.010 INFO:teuthology.orchestra.run.smithi049.stdout:2060 writesame 0xd487673 thru 0xd48b1d1 data_size 0x1(0x3b5f bytes) 2024-04-24T18:08:09.020 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.020 INFO:teuthology.orchestra.run.smithi049.stdout:2065 write 0x773cf7a thru 0x7744e58 (0x7edf bytes) 2024-04-24T18:08:09.023 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.024 INFO:teuthology.orchestra.run.smithi049.stdout:2067 write 0x4828852 thru 0x482a09e (0x184d bytes) 2024-04-24T18:08:09.034 INFO:teuthology.orchestra.run.smithi049.stdout:2068 trunc from 0xd59e249 to 0x60ea8cc 2024-04-24T18:08:09.038 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:09.038 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:09.044 INFO:teuthology.orchestra.run.smithi049.stdout:2070 read 0x190c5a4 thru 0x191237a (0x5dd7 bytes) 2024-04-24T18:08:09.044 INFO:teuthology.orchestra.run.smithi049.stdout:2071 read 0x25d4cbe thru 0x25de121 (0x9464 bytes) 2024-04-24T18:08:09.045 INFO:teuthology.orchestra.run.smithi049.stdout:2074 writesame 0x2077e06 thru 0x207f5e5 data_size 0x20(0x77e0 bytes) 2024-04-24T18:08:09.048 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.048 INFO:teuthology.orchestra.run.smithi049.stdout:2076 write 0x48278f8 thru 0x4830ff3 (0x96fc bytes) 2024-04-24T18:08:09.049 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:09.062 INFO:teuthology.orchestra.run.smithi049.stdout:2077 write 0xbebb1f2 thru 0xbebbf81 (0xd90 bytes) 2024-04-24T18:08:09.066 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.066 INFO:teuthology.orchestra.run.smithi049.stdout:2079 read 0x93b7af1 thru 0x93b7ed6 (0x3e6 bytes) 2024-04-24T18:08:09.066 INFO:teuthology.orchestra.run.smithi049.stdout:2080 writesame 0x96dee95 thru 0x96edaeb data_size 0x1(0xec57 bytes) 2024-04-24T18:08:09.097 INFO:teuthology.orchestra.run.smithi049.stdout:2081 read 0x4791911 thru 0x4791f59 (0x649 bytes) 2024-04-24T18:08:09.097 INFO:teuthology.orchestra.run.smithi049.stdout:2082 punch from 0x87ff607 to 0x88093f2, (0x9deb bytes) 2024-04-24T18:08:09.097 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.097 INFO:teuthology.orchestra.run.smithi049.stdout:2084 read 0x42300dc thru 0x4237d5b (0x7c80 bytes) 2024-04-24T18:08:09.102 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:09.110+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:09.104 INFO:teuthology.orchestra.run.smithi049.stdout:2085 writesame 0xe40d8ef thru 0xe411dab data_size 0x1(0x44bd bytes) 2024-04-24T18:08:09.104 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.105 INFO:teuthology.orchestra.run.smithi049.stdout:2087 trunc from 0xe411dac to 0xd20853e 2024-04-24T18:08:09.110 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.110 INFO:teuthology.orchestra.run.smithi049.stdout:2090 punch from 0x2ea919b to 0x2eb203a, (0x8e9f bytes) 2024-04-24T18:08:09.111 INFO:teuthology.orchestra.run.smithi049.stdout:2091 write 0x3f473a0 thru 0x3f50c48 (0x98a9 bytes) 2024-04-24T18:08:09.113 INFO:teuthology.orchestra.run.smithi049.stdout:2092 read 0x86a490f thru 0x86acc00 (0x82f2 bytes) 2024-04-24T18:08:09.114 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.114 INFO:teuthology.orchestra.run.smithi049.stdout:2094 writesame 0x1026a9b thru 0x102e34e data_size 0x3(0x78b4 bytes) 2024-04-24T18:08:09.120 INFO:teuthology.orchestra.run.smithi049.stdout:2095 write 0x3bfbc8e thru 0x3c01af3 (0x5e66 bytes) 2024-04-24T18:08:09.124 INFO:teuthology.orchestra.run.smithi049.stdout:2096 read 0x6a75882 thru 0x6a829bb (0xd13a bytes) 2024-04-24T18:08:09.124 INFO:teuthology.orchestra.run.smithi049.stdout:2097 write 0x33f970d thru 0x34029d9 (0x92cd bytes) 2024-04-24T18:08:09.127 INFO:teuthology.orchestra.run.smithi049.stdout:2099 read 0x184f3fd thru 0x1857642 (0x8246 bytes) 2024-04-24T18:08:09.128 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.128 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.128 INFO:teuthology.orchestra.run.smithi049.stdout:2104 punch from 0xbb1dde1 to 0xbb258b5, (0x7ad4 bytes) 2024-04-24T18:08:09.128 INFO:teuthology.orchestra.run.smithi049.stdout:2106 writesame 0xbca1fa1 thru 0xbca2635 data_size 0x1(0x695 bytes) 2024-04-24T18:08:09.131 INFO:teuthology.orchestra.run.smithi049.stdout:2108 write 0xb65fad thru 0xb71b4e (0xbba2 bytes) 2024-04-24T18:08:09.135 INFO:teuthology.orchestra.run.smithi049.stdout:2109 write 0xd037b9b thru 0xd03a64c (0x2ab2 bytes) 2024-04-24T18:08:09.137 INFO:teuthology.orchestra.run.smithi049.stdout:2111 write 0xb2478b7 thru 0xb24e665 (0x6daf bytes) 2024-04-24T18:08:09.140 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.140 INFO:teuthology.orchestra.run.smithi049.stdout:2113 read 0xc59f921 thru 0xc5af3b1 (0xfa91 bytes) 2024-04-24T18:08:09.146 INFO:teuthology.orchestra.run.smithi049.stdout:2114 writesame 0xdd0c75a thru 0xdd12612 data_size 0x3b(0x5eb9 bytes) 2024-04-24T18:08:09.146 INFO:teuthology.orchestra.run.smithi049.stdout:2115 writesame 0x1b8ab48 thru 0x1b974dc data_size 0x1(0xc995 bytes) 2024-04-24T18:08:09.174 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.181 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:09.181 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:09.186 INFO:teuthology.orchestra.run.smithi049.stdout:2118 trunc from 0xdd12613 to 0x37a7562 2024-04-24T18:08:09.192 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:09.203 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:09.211+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:09.212 INFO:teuthology.orchestra.run.smithi049.stdout:2119 write 0xc588ce0 thru 0xc594175 (0xb496 bytes) 2024-04-24T18:08:09.215 INFO:teuthology.orchestra.run.smithi049.stdout:2121 punch from 0x5712fcf to 0x571e6a3, (0xb6d4 bytes) 2024-04-24T18:08:09.215 INFO:teuthology.orchestra.run.smithi049.stdout:2122 writesame 0x222d226 thru 0x2233760 data_size 0x47(0x653b bytes) 2024-04-24T18:08:09.218 INFO:teuthology.orchestra.run.smithi049.stdout:2123 write 0x833e269 thru 0x83457cf (0x7567 bytes) 2024-04-24T18:08:09.222 INFO:teuthology.orchestra.run.smithi049.stdout:2126 punch from 0x483517f to 0x4838ba3, (0x3a24 bytes) 2024-04-24T18:08:09.222 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:09.222 INFO:teuthology.orchestra.run.smithi049.stdout:2128 read 0xd18a4d thru 0xd24404 (0xb9b8 bytes) 2024-04-24T18:08:09.223 INFO:teuthology.orchestra.run.smithi049.stdout:2129 writesame 0x3108aff thru 0x31185a8 data_size 0x5(0xfaaa bytes) 2024-04-24T18:08:09.231 INFO:teuthology.orchestra.run.smithi049.stdout:2130 clone 19 order 20 su 131072 sc 10 2024-04-24T18:08:09.304 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:09.313+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:09.317 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:09.317 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:09.328 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:09.405 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:09.414+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:09.452 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:09.452 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:09.462 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:08:09.506 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:09.514+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:09.606 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:09.614+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:09.626 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:09.627 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:09.636 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:08:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.4 to 1 (10000) 2024-04-24T18:08:09.672 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:08:09.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-24T18:08:09.730 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:09.738+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:09.741 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone18 from 0xc594176 (overlap 0x29aeddb) to 0x2583c78 2024-04-24T18:08:09.767 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:09.768 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:09.778 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:09.807 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:09.815+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:09.853 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #17, image image_client.0-clone17 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent18 2024-04-24T18:08:09.904 INFO:teuthology.orchestra.run.smithi049.stdout:2131 trunc from 0xc594176 to 0x9d4af5b 2024-04-24T18:08:09.905 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:09.905 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:09.908 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:09.916+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:09.915 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:09.924 INFO:teuthology.orchestra.run.smithi049.stdout:2132 read 0x18600b3 thru 0x186df2e (0xde7c bytes) 2024-04-24T18:08:09.925 INFO:teuthology.orchestra.run.smithi049.stdout:2133 write 0x98b792e thru 0x98c004b (0x871e bytes) 2024-04-24T18:08:09.926 INFO:teuthology.orchestra.run.smithi049.stdout:2134 read 0x12dc07 thru 0x1336d0 (0x5aca bytes) 2024-04-24T18:08:09.927 INFO:teuthology.orchestra.run.smithi049.stdout:2135 read 0x1186b0c thru 0x1194631 (0xdb26 bytes) 2024-04-24T18:08:09.927 INFO:teuthology.orchestra.run.smithi049.stdout:2136 write 0x39c5e2 thru 0x3a3164 (0x6b83 bytes) 2024-04-24T18:08:09.932 INFO:teuthology.orchestra.run.smithi049.stdout:2137 punch from 0x824b888 to 0x8255971, (0xa0e9 bytes) 2024-04-24T18:08:09.932 INFO:teuthology.orchestra.run.smithi049.stdout:2138 read 0x24d5f2e thru 0x24df0ac (0x917f bytes) 2024-04-24T18:08:09.940 INFO:teuthology.orchestra.run.smithi049.stdout:2139 trunc from 0x9d4af5b to 0x3898453 2024-04-24T18:08:09.972 INFO:teuthology.orchestra.run.smithi049.stdout:2140 write 0x88d5685 thru 0x88d5749 (0xc5 bytes) 2024-04-24T18:08:09.981 INFO:teuthology.orchestra.run.smithi049.stdout:2141 write 0x90b2e4f thru 0x90b9efa (0x70ac bytes) 2024-04-24T18:08:09.994 INFO:teuthology.orchestra.run.smithi049.stdout:2142 write 0xd9766de thru 0xd97a98a (0x42ad bytes) 2024-04-24T18:08:09.997 INFO:teuthology.orchestra.run.smithi049.stdout:2144 read 0x15747bf thru 0x157cff9 (0x883b bytes) 2024-04-24T18:08:10.009 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:10.017+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:10.012 INFO:teuthology.orchestra.run.smithi049.stdout:2145 trunc from 0xd97a98b to 0xf453f7 2024-04-24T18:08:10.037 INFO:teuthology.orchestra.run.smithi049.stdout:2146 read 0x74dea6 thru 0x75a089 (0xc1e4 bytes) 2024-04-24T18:08:10.037 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.043 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:10.043 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:10.046 INFO:teuthology.orchestra.run.smithi049.stdout:2148 trunc from 0xf453f7 to 0x8e3483a 2024-04-24T18:08:10.051 INFO:teuthology.orchestra.run.smithi049.stdout:2149 read 0x5348ef2 thru 0x5357f5e (0xf06d bytes) 2024-04-24T18:08:10.051 INFO:teuthology.orchestra.run.smithi049.stdout:2151 write 0x82661b0 thru 0x826b9de (0x582f bytes) 2024-04-24T18:08:10.054 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:10.058 INFO:teuthology.orchestra.run.smithi049.stdout:2152 trunc from 0x8e3483a to 0xd3d08e1 2024-04-24T18:08:10.064 INFO:teuthology.orchestra.run.smithi049.stdout:2153 punch from 0x402cc90 to 0x4034f48, (0x82b8 bytes) 2024-04-24T18:08:10.064 INFO:teuthology.orchestra.run.smithi049.stdout:2155 write 0x41fe403 thru 0x4209718 (0xb316 bytes) 2024-04-24T18:08:10.068 INFO:teuthology.orchestra.run.smithi049.stdout:2156 write 0xc8e7b2f thru 0xc8eea00 (0x6ed2 bytes) 2024-04-24T18:08:10.070 INFO:teuthology.orchestra.run.smithi049.stdout:2158 trunc from 0xd3d08e1 to 0xd1e3dd7 2024-04-24T18:08:10.075 INFO:teuthology.orchestra.run.smithi049.stdout:2159 write 0x3597537 thru 0x359eb8e (0x7658 bytes) 2024-04-24T18:08:10.078 INFO:teuthology.orchestra.run.smithi049.stdout:2161 punch from 0x7436c48 to 0x7445d40, (0xf0f8 bytes) 2024-04-24T18:08:10.092 INFO:teuthology.orchestra.run.smithi049.stdout:2162 trunc from 0xd1e3dd7 to 0x904e81 2024-04-24T18:08:10.110 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:10.118+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:10.132 INFO:teuthology.orchestra.run.smithi049.stdout:2163 write 0xa33e95c thru 0xa34774e (0x8df3 bytes) 2024-04-24T18:08:10.136 INFO:teuthology.orchestra.run.smithi049.stdout:2164 read 0x10d9d31 thru 0x10dacb7 (0xf87 bytes) 2024-04-24T18:08:10.136 INFO:teuthology.orchestra.run.smithi049.stdout:2165 read 0x8b3c224 thru 0x8b43fcf (0x7dac bytes) 2024-04-24T18:08:10.144 INFO:teuthology.orchestra.run.smithi049.stdout:2166 writesame 0xca19ac8 thru 0xca2557d data_size 0x1(0xbab6 bytes) 2024-04-24T18:08:10.144 INFO:teuthology.orchestra.run.smithi049.stdout:2168 punch from 0x6be9258 to 0x6bf5164, (0xbf0c bytes) 2024-04-24T18:08:10.144 INFO:teuthology.orchestra.run.smithi049.stdout:2169 punch from 0x425a061 to 0x425faaf, (0x5a4e bytes) 2024-04-24T18:08:10.144 INFO:teuthology.orchestra.run.smithi049.stdout:2173 read 0xd33c83 thru 0xd37c45 (0x3fc3 bytes) 2024-04-24T18:08:10.145 INFO:teuthology.orchestra.run.smithi049.stdout:2174 punch from 0x7551f4d to 0x755faa7, (0xdb5a bytes) 2024-04-24T18:08:10.145 INFO:teuthology.orchestra.run.smithi049.stdout:2175 write 0x4dfbfa6 thru 0x4e003ec (0x4447 bytes) 2024-04-24T18:08:10.147 INFO:teuthology.orchestra.run.smithi049.stdout:2176 write 0x17dced3 thru 0x17dffe1 (0x310f bytes) 2024-04-24T18:08:10.150 INFO:teuthology.orchestra.run.smithi049.stdout:2177 write 0x23ff7f7 thru 0x2405532 (0x5d3c bytes) 2024-04-24T18:08:10.155 INFO:teuthology.orchestra.run.smithi049.stdout:2178 trunc from 0xca2557e to 0xe596716 2024-04-24T18:08:10.160 INFO:teuthology.orchestra.run.smithi049.stdout:2179 write 0x8b7f873 thru 0x8b89ff3 (0xa781 bytes) 2024-04-24T18:08:10.163 INFO:teuthology.orchestra.run.smithi049.stdout:2180 writesame 0x3f39502 thru 0x3f44649 data_size 0x7c(0xb148 bytes) 2024-04-24T18:08:10.167 INFO:teuthology.orchestra.run.smithi049.stdout:2181 trunc from 0xe596716 to 0xe99fa19 2024-04-24T18:08:10.172 INFO:teuthology.orchestra.run.smithi049.stdout:2184 read 0xd96d8d2 thru 0xd96f13c (0x186b bytes) 2024-04-24T18:08:10.172 INFO:teuthology.orchestra.run.smithi049.stdout:2187 read 0xa53ea4b thru 0xa54ddc1 (0xf377 bytes) 2024-04-24T18:08:10.172 INFO:teuthology.orchestra.run.smithi049.stdout:2188 read 0x80c6c4f thru 0x80c7261 (0x613 bytes) 2024-04-24T18:08:10.172 INFO:teuthology.orchestra.run.smithi049.stdout:2189 write 0xaad0904 thru 0xaad7c13 (0x7310 bytes) 2024-04-24T18:08:10.175 INFO:teuthology.orchestra.run.smithi049.stdout:2190 writesame 0x1218ca8 thru 0x1223345 data_size 0x3(0xa69e bytes) 2024-04-24T18:08:10.182 INFO:teuthology.orchestra.run.smithi049.stdout:2191 punch from 0xc8ee5b9 to 0xc8f7fb7, (0x99fe bytes) 2024-04-24T18:08:10.182 INFO:teuthology.orchestra.run.smithi049.stdout:2192 read 0xcfbf266 thru 0xcfc11ab (0x1f46 bytes) 2024-04-24T18:08:10.182 INFO:teuthology.orchestra.run.smithi049.stdout:2193 read 0x224933c thru 0x224fb9a (0x685f bytes) 2024-04-24T18:08:10.182 INFO:teuthology.orchestra.run.smithi049.stdout:2194 writesame 0x83b220f thru 0x83ba9a3 data_size 0x1(0x8795 bytes) 2024-04-24T18:08:10.185 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:10.185 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:10.196 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:10.199 INFO:teuthology.orchestra.run.smithi049.stdout:2196 punch from 0xca383e to 0xca5df3, (0x25b5 bytes) 2024-04-24T18:08:10.200 INFO:teuthology.orchestra.run.smithi049.stdout:2197 punch from 0x7ff2902 to 0x7ff8622, (0x5d20 bytes) 2024-04-24T18:08:10.200 INFO:teuthology.orchestra.run.smithi049.stdout:2198 write 0x7953068 thru 0x795b366 (0x82ff bytes) 2024-04-24T18:08:10.203 INFO:teuthology.orchestra.run.smithi049.stdout:2200 writesame 0x134653a thru 0x1349b6e data_size 0x1(0x3635 bytes) 2024-04-24T18:08:10.210 INFO:teuthology.orchestra.run.smithi049.stdout:2201 write 0x78bf665 thru 0x78c4608 (0x4fa4 bytes) 2024-04-24T18:08:10.211 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:10.219+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:10.215 INFO:teuthology.orchestra.run.smithi049.stdout:2202 read 0x32e5da9 thru 0x32f38f0 (0xdb48 bytes) 2024-04-24T18:08:10.215 INFO:teuthology.orchestra.run.smithi049.stdout:2203 write 0x1bf8e9f thru 0x1bfbad1 (0x2c33 bytes) 2024-04-24T18:08:10.217 INFO:teuthology.orchestra.run.smithi049.stdout:2204 writesame 0xd485812 thru 0xd491a11 data_size 0x80(0xc200 bytes) 2024-04-24T18:08:10.221 INFO:teuthology.orchestra.run.smithi049.stdout:2205 punch from 0x14ef6f6 to 0x14fcffe, (0xd908 bytes) 2024-04-24T18:08:10.221 INFO:teuthology.orchestra.run.smithi049.stdout:2206 write 0x825c052 thru 0x8262cbf (0x6c6e bytes) 2024-04-24T18:08:10.224 INFO:teuthology.orchestra.run.smithi049.stdout:2207 write 0xb4d261f thru 0xb4d5179 (0x2b5b bytes) 2024-04-24T18:08:10.227 INFO:teuthology.orchestra.run.smithi049.stdout:2209 write 0x56633bd thru 0x5669d2e (0x6972 bytes) 2024-04-24T18:08:10.230 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.230 INFO:teuthology.orchestra.run.smithi049.stdout:2211 read 0x5cdfa30 thru 0x5cebc5d (0xc22e bytes) 2024-04-24T18:08:10.230 INFO:teuthology.orchestra.run.smithi049.stdout:2214 writesame 0x9d477df thru 0x9d48e1f data_size 0x9(0x1641 bytes) 2024-04-24T18:08:10.233 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.233 INFO:teuthology.orchestra.run.smithi049.stdout:2216 punch from 0x775e03b to 0x776461c, (0x65e1 bytes) 2024-04-24T18:08:10.233 INFO:teuthology.orchestra.run.smithi049.stdout:2217 punch from 0x637d368 to 0x638c8ae, (0xf546 bytes) 2024-04-24T18:08:10.233 INFO:teuthology.orchestra.run.smithi049.stdout:2218 write 0x14f636d thru 0x14fffec (0x9c80 bytes) 2024-04-24T18:08:10.236 INFO:teuthology.orchestra.run.smithi049.stdout:2220 write 0x8f6a5db thru 0x8f6a9f6 (0x41c bytes) 2024-04-24T18:08:10.238 INFO:teuthology.orchestra.run.smithi049.stdout:2221 writesame 0x5563e4c thru 0x556a24a data_size 0x45(0x63ff bytes) 2024-04-24T18:08:10.241 INFO:teuthology.orchestra.run.smithi049.stdout:2222 write 0xcca1361 thru 0xcca7168 (0x5e08 bytes) 2024-04-24T18:08:10.244 INFO:teuthology.orchestra.run.smithi049.stdout:2224 read 0xe55d498 thru 0xe56b873 (0xe3dc bytes) 2024-04-24T18:08:10.244 INFO:teuthology.orchestra.run.smithi049.stdout:2225 write 0xd94a16f thru 0xd94e426 (0x42b8 bytes) 2024-04-24T18:08:10.246 INFO:teuthology.orchestra.run.smithi049.stdout:2226 write 0xa3805d9 thru 0xa38cb3b (0xc563 bytes) 2024-04-24T18:08:10.251 INFO:teuthology.orchestra.run.smithi049.stdout:2227 trunc from 0xe99fa19 to 0xd69a398 2024-04-24T18:08:10.257 INFO:teuthology.orchestra.run.smithi049.stdout:2228 writesame 0xbd6856e thru 0xbd6adb7 data_size 0x1b(0x284a bytes) 2024-04-24T18:08:10.260 INFO:teuthology.orchestra.run.smithi049.stdout:2229 read 0x7735da thru 0x778d06 (0x572d bytes) 2024-04-24T18:08:10.260 INFO:teuthology.orchestra.run.smithi049.stdout:2231 write 0x9b4ead8 thru 0x9b5b73c (0xcc65 bytes) 2024-04-24T18:08:10.263 INFO:teuthology.orchestra.run.smithi049.stdout:2232 writesame 0x3cc16b7 thru 0x3ccdba1 data_size 0x1(0xc4eb bytes) 2024-04-24T18:08:10.288 INFO:teuthology.orchestra.run.smithi049.stdout:2233 write 0x26dc5ca thru 0x26e7d14 (0xb74b bytes) 2024-04-24T18:08:10.292 INFO:teuthology.orchestra.run.smithi049.stdout:2236 writesame 0x8e4063e thru 0x8e42a4e data_size 0x1(0x2411 bytes) 2024-04-24T18:08:10.298 INFO:teuthology.orchestra.run.smithi049.stdout:2237 write 0x7cc51dc thru 0x7cd1704 (0xc529 bytes) 2024-04-24T18:08:10.301 INFO:teuthology.orchestra.run.smithi049.stdout:2238 read 0x7840237 thru 0x7843dbd (0x3b87 bytes) 2024-04-24T18:08:10.301 INFO:teuthology.orchestra.run.smithi049.stdout:2239 write 0x3fef42b thru 0x3fef6a1 (0x277 bytes) 2024-04-24T18:08:10.303 INFO:teuthology.orchestra.run.smithi049.stdout:2240 read 0x85e5597 thru 0x85e6424 (0xe8e bytes) 2024-04-24T18:08:10.304 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.304 INFO:teuthology.orchestra.run.smithi049.stdout:2242 read 0xa646a52 thru 0xa6479d3 (0xf82 bytes) 2024-04-24T18:08:10.304 INFO:teuthology.orchestra.run.smithi049.stdout:2243 writesame 0x3a0bf0d thru 0x3a0eb25 data_size 0x35(0x2c19 bytes) 2024-04-24T18:08:10.306 INFO:teuthology.orchestra.run.smithi049.stdout:2245 writesame 0x26b9eeb thru 0x26beae0 data_size 0x15(0x4bf6 bytes) 2024-04-24T18:08:10.309 INFO:teuthology.orchestra.run.smithi049.stdout:2246 punch from 0x3043a4e to 0x3044788, (0xd3a bytes) 2024-04-24T18:08:10.310 INFO:teuthology.orchestra.run.smithi049.stdout:2247 punch from 0xc27f969 to 0xc28b7e1, (0xbe78 bytes) 2024-04-24T18:08:10.310 INFO:teuthology.orchestra.run.smithi049.stdout:2248 write 0x1d8e98c thru 0x1d93e6b (0x54e0 bytes) 2024-04-24T18:08:10.312 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:10.320+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:10.313 INFO:teuthology.orchestra.run.smithi049.stdout:2249 write 0x90c85eb thru 0x90d79af (0xf3c5 bytes) 2024-04-24T18:08:10.314 INFO:teuthology.orchestra.run.smithi049.stdout:2250 read 0x50535f0 thru 0x5053a0d (0x41e bytes) 2024-04-24T18:08:10.315 INFO:teuthology.orchestra.run.smithi049.stdout:2251 read 0xe0e004 thru 0xe1a386 (0xc383 bytes) 2024-04-24T18:08:10.315 INFO:teuthology.orchestra.run.smithi049.stdout:2252 writesame 0x39a5870 thru 0x39a5a95 data_size 0xb(0x226 bytes) 2024-04-24T18:08:10.317 INFO:teuthology.orchestra.run.smithi049.stdout:2253 write 0x2c19b0c thru 0x2c1c820 (0x2d15 bytes) 2024-04-24T18:08:10.320 INFO:teuthology.orchestra.run.smithi049.stdout:2254 read 0x39b6e35 thru 0x39bd910 (0x6adc bytes) 2024-04-24T18:08:10.321 INFO:teuthology.orchestra.run.smithi049.stdout:2255 punch from 0x94299d2 to 0x943320c, (0x983a bytes) 2024-04-24T18:08:10.321 INFO:teuthology.orchestra.run.smithi049.stdout:2256 write 0x84d4273 thru 0x84dcd98 (0x8b26 bytes) 2024-04-24T18:08:10.322 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.333 INFO:teuthology.orchestra.run.smithi049.stdout:2258 trunc from 0xd69a398 to 0x32b1db7 2024-04-24T18:08:10.341 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:10.341 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:10.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:10.366 INFO:teuthology.orchestra.run.smithi049.stdout:2260 writesame 0xd1631b9 thru 0xd16d1b9 data_size 0x1(0xa001 bytes) 2024-04-24T18:08:10.373 INFO:teuthology.orchestra.run.smithi049.stdout:2261 write 0xe9fb492 thru 0xea01828 (0x6397 bytes) 2024-04-24T18:08:10.390 INFO:teuthology.orchestra.run.smithi049.stdout:2264 trunc from 0xea01829 to 0x2cb746d 2024-04-24T18:08:10.401 INFO:teuthology.orchestra.run.smithi049.stdout:2265 read 0xd635c thru 0xd8e6c (0x2b11 bytes) 2024-04-24T18:08:10.402 INFO:teuthology.orchestra.run.smithi049.stdout:2266 writesame 0x296b975 thru 0x2971efd data_size 0x11(0x6589 bytes) 2024-04-24T18:08:10.405 INFO:teuthology.orchestra.run.smithi049.stdout:2267 read 0x12bf665 thru 0x12c2b80 (0x351c bytes) 2024-04-24T18:08:10.405 INFO:teuthology.orchestra.run.smithi049.stdout:2270 punch from 0x1bdb0f8 to 0x1be5065, (0x9f6d bytes) 2024-04-24T18:08:10.412 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:10.420+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:10.417 INFO:teuthology.orchestra.run.smithi049.stdout:2272 write 0x890b675 thru 0x8911dc2 (0x674e bytes) 2024-04-24T18:08:10.419 INFO:teuthology.orchestra.run.smithi049.stdout:2273 writesame 0x366927e thru 0x36694c2 data_size 0x1(0x245 bytes) 2024-04-24T18:08:10.422 INFO:teuthology.orchestra.run.smithi049.stdout:2275 trunc from 0x8911dc3 to 0x8e37b15 2024-04-24T18:08:10.427 INFO:teuthology.orchestra.run.smithi049.stdout:2276 read 0x6d3361f thru 0x6d36495 (0x2e77 bytes) 2024-04-24T18:08:10.436 INFO:teuthology.orchestra.run.smithi049.stdout:2277 write 0xcad675e thru 0xcae1c5f (0xb502 bytes) 2024-04-24T18:08:10.440 INFO:teuthology.orchestra.run.smithi049.stdout:2278 writesame 0x17ddcd3 thru 0x17e2362 data_size 0x8(0x4690 bytes) 2024-04-24T18:08:10.443 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.444 INFO:teuthology.orchestra.run.smithi049.stdout:2282 write 0xabdde56 thru 0xabdea6f (0xc1a bytes) 2024-04-24T18:08:10.446 INFO:teuthology.orchestra.run.smithi049.stdout:2284 write 0x8dfa87a thru 0x8e057ed (0xaf74 bytes) 2024-04-24T18:08:10.450 INFO:teuthology.orchestra.run.smithi049.stdout:2285 write 0x85ded21 thru 0x85e60fa (0x73da bytes) 2024-04-24T18:08:10.453 INFO:teuthology.orchestra.run.smithi049.stdout:2287 read 0xc6ad655 thru 0xc6bb12c (0xdad8 bytes) 2024-04-24T18:08:10.460 INFO:teuthology.orchestra.run.smithi049.stdout:2288 write 0xe22884f thru 0xe22e6f1 (0x5ea3 bytes) 2024-04-24T18:08:10.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:10.474 INFO:teuthology.orchestra.run.smithi049.stdout:2289 trunc from 0xe22e6f2 to 0x46049c5 2024-04-24T18:08:10.486 INFO:teuthology.orchestra.run.smithi049.stdout:2291 read 0x3fd71a1 thru 0x3fde670 (0x74d0 bytes) 2024-04-24T18:08:10.486 INFO:teuthology.orchestra.run.smithi049.stdout:2293 read 0x429d79a thru 0x42aaefe (0xd765 bytes) 2024-04-24T18:08:10.499 INFO:teuthology.orchestra.run.smithi049.stdout:2295 write 0x94ef985 thru 0x94f0520 (0xb9c bytes) 2024-04-24T18:08:10.502 INFO:teuthology.orchestra.run.smithi049.stdout:2296 punch from 0x5b72c8c to 0x5b769f6, (0x3d6a bytes) 2024-04-24T18:08:10.502 INFO:teuthology.orchestra.run.smithi049.stdout:2297 read 0x9d1d5f thru 0x9d6d62 (0x5004 bytes) 2024-04-24T18:08:10.502 INFO:teuthology.orchestra.run.smithi049.stdout:2298 punch from 0x46aa7f7 to 0x46b2f29, (0x8732 bytes) 2024-04-24T18:08:10.512 INFO:teuthology.orchestra.run.smithi049.stdout:2299 write 0xc28636f thru 0xc294f2d (0xebbf bytes) 2024-04-24T18:08:10.513 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:10.521+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:10.515 INFO:teuthology.orchestra.run.smithi049.stdout:2301 write 0x76a5220 thru 0x76a8551 (0x3332 bytes) 2024-04-24T18:08:10.524 INFO:teuthology.orchestra.run.smithi049.stdout:2302 write 0xc9c7c0f thru 0xc9c989d (0x1c8f bytes) 2024-04-24T18:08:10.527 INFO:teuthology.orchestra.run.smithi049.stdout:2303 writesame 0x32865d6 thru 0x3290d68 data_size 0x1(0xa793 bytes) 2024-04-24T18:08:10.529 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:10.529 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:10.543 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:10.551 INFO:teuthology.orchestra.run.smithi049.stdout:2305 punch from 0x4b42af9 to 0x4b510f3, (0xe5fa bytes) 2024-04-24T18:08:10.552 INFO:teuthology.orchestra.run.smithi049.stdout:2308 writesame 0x144bd9b thru 0x14502ff data_size 0x55(0x4565 bytes) 2024-04-24T18:08:10.566 INFO:teuthology.orchestra.run.smithi049.stdout:2309 trunc from 0xc9c989e to 0x1ebf7fd 2024-04-24T18:08:10.585 INFO:teuthology.orchestra.run.smithi049.stdout:2314 punch from 0x48f27b to 0x4930c3, (0x3e48 bytes) 2024-04-24T18:08:10.597 INFO:teuthology.orchestra.run.smithi049.stdout:2315 writesame 0x69483ea thru 0x69581b4 data_size 0x9(0xfdcb bytes) 2024-04-24T18:08:10.605 INFO:teuthology.orchestra.run.smithi049.stdout:2316 write 0x89e5e05 thru 0x89f3b3b (0xdd37 bytes) 2024-04-24T18:08:10.608 INFO:teuthology.orchestra.run.smithi049.stdout:2317 write 0x10a499c thru 0x10a779a (0x2dff bytes) 2024-04-24T18:08:10.611 INFO:teuthology.orchestra.run.smithi049.stdout:2321 punch from 0x1c5dcd4 to 0x1c605a3, (0x28cf bytes) 2024-04-24T18:08:10.613 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:10.622+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:10.619 INFO:teuthology.orchestra.run.smithi049.stdout:2323 write 0xa75cfd6 thru 0xa760563 (0x358e bytes) 2024-04-24T18:08:10.625 INFO:teuthology.orchestra.run.smithi049.stdout:2325 read 0x7b4e6ea thru 0x7b4feef (0x1806 bytes) 2024-04-24T18:08:10.625 INFO:teuthology.orchestra.run.smithi049.stdout:2326 read 0x646fac9 thru 0x647d127 (0xd65f bytes) 2024-04-24T18:08:10.625 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.626 INFO:teuthology.orchestra.run.smithi049.stdout:2328 writesame 0x2a00376 thru 0x2a02c28 data_size 0x17(0x28b3 bytes) 2024-04-24T18:08:10.629 INFO:teuthology.orchestra.run.smithi049.stdout:2330 read 0x12e2cf3 thru 0x12ec0c1 (0x93cf bytes) 2024-04-24T18:08:10.629 INFO:teuthology.orchestra.run.smithi049.stdout:2331 punch from 0x6856bb1 to 0x6863445, (0xc894 bytes) 2024-04-24T18:08:10.629 INFO:teuthology.orchestra.run.smithi049.stdout:2332 read 0x4a31cb6 thru 0x4a40d60 (0xf0ab bytes) 2024-04-24T18:08:10.629 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.629 INFO:teuthology.orchestra.run.smithi049.stdout:2334 read 0x216b04a thru 0x216edc1 (0x3d78 bytes) 2024-04-24T18:08:10.629 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.629 INFO:teuthology.orchestra.run.smithi049.stdout:2336 clone 20 order 18 su 1024 sc 4 2024-04-24T18:08:10.644 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:10.663 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:10.676 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.5 to 1 (10000) 2024-04-24T18:08:10.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-24T18:08:10.694 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:10.719 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:10.719 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:10.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:10.719 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:10.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31810 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 2920448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.033533423000002, 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052993206000000001, 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31810 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:10.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691739+0000", 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691742+0000", 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2800000000000002e-06 2024-04-24T18:08:10.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691747+0000", 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4319999999999998e-06 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691750+0000", 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.247e-06 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744520+0000", 2024-04-24T18:08:10.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052769498999999997 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744623+0000", 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000102808 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744733+0000", 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110282 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.019948663000001, 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:10.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:08:10.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:08:10.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:10.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32297 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3233792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.00757458, 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052981070999999998, 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.729 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32297 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717698+0000", 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717705+0000", 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5529999999999998e-06 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717711+0000", 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3849999999999999e-06 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717715+0000", 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770506+0000", 2024-04-24T18:08:10.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052791267000000003 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770587+0000", 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0783000000000002e-05 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770680+0000", 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2497000000000006e-05 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32299 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.007553576999999, 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053161606, 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32299 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717719+0000", 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717722+0000", 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.43e-06 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717729+0000", 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.576e-06 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717733+0000", 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0119999999999999e-06 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770707+0000", 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052974083999999998 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770791+0000", 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3418000000000007e-05 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770881+0000", 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0490999999999998e-05 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32301 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.007535706999999, 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053336189999999999, 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32301 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.738 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717737+0000", 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717740+0000", 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4090000000000001e-06 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717746+0000", 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9989999999999997e-06 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717752+0000", 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0349999999999998e-06 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770915+0000", 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053162938 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770989+0000", 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4567999999999995e-05 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771074+0000", 2024-04-24T18:08:10.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4598000000000006e-05 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32303 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3270656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.007422629000001, 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053426976000000001, 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32303 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717850+0000", 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717854+0000", 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0509999999999998e-06 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717863+0000", 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.346e-06 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717877+0000", 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3559e-05 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771102+0000", 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053224869000000001 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771184+0000", 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2564000000000004e-05 2024-04-24T18:08:10.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771278+0000", 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3178999999999997e-05 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32305 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3278848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.007390461, 2024-04-24T18:08:10.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053587411000000001, 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32305 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717882+0000", 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717885+0000", 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8679999999999999e-06 2024-04-24T18:08:10.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717892+0000", 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717896+0000", 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9360000000000001e-06 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771305+0000", 2024-04-24T18:08:10.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053409310000000002 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771373+0000", 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8417999999999994e-05 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771470+0000", 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6827e-05 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.750 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32307 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3289088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.00737342, 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053767057, 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32307 2024-04-24T18:08:10.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717899+0000", 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717902+0000", 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5019999999999999e-06 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717908+0000", 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7699999999999998e-06 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717911+0000", 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4979999999999998e-06 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771498+0000", 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053586747999999997 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771571+0000", 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3168000000000001e-05 2024-04-24T18:08:10.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771667+0000", 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5810000000000006e-05 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32311 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3307520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.006077035000001, 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053356705999999997, 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.755 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32311 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719199+0000", 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.88e-06 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719205+0000", 2024-04-24T18:08:10.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719209+0000", 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0749999999999999e-06 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772008+0000", 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052798572000000002 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772048+0000", 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0238999999999999e-05 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772553+0000", 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050474099999999996 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32313 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3315712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.006058976999999, 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053105309000000003, 2024-04-24T18:08:10.759 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32313 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:10.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719216+0000", 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719223+0000", 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3010000000000003e-06 2024-04-24T18:08:10.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719226+0000", 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7610000000000001e-06 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772130+0000", 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052903238999999998 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772209+0000", 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9486000000000004e-05 2024-04-24T18:08:10.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772320+0000", 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011052099999999999 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32315 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3334144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.006042334, 2024-04-24T18:08:10.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053250991999999997, 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32315 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719233+0000", 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1789999999999998e-06 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719240+0000", 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5689999999999999e-06 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719244+0000", 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5870000000000001e-06 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772345+0000", 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053100972000000003 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772413+0000", 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8071000000000006e-05 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772482+0000", 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8950999999999998e-05 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32317 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.005960272999999, 2024-04-24T18:08:10.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053370316000000001, 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32317 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719312+0000", 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719316+0000", 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9400000000000004e-06 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719326+0000", 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2439999999999999e-06 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719331+0000", 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7049999999999996e-06 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772583+0000", 2024-04-24T18:08:10.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053251860999999998 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772628+0000", 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5289e-05 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772683+0000", 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4882000000000001e-05 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32319 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3352576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.005936930000001, 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053459060000000003, 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32319 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:10.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719338+0000", 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5490000000000001e-06 2024-04-24T18:08:10.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719346+0000", 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5059999999999998e-06 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719350+0000", 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1999999999999996e-06 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772709+0000", 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053359182999999998 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772748+0000", 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8654e-05 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772795+0000", 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7437e-05 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32321 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3360768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.00591872, 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053627222000000002, 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32321 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719354+0000", 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719361+0000", 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7449999999999998e-06 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719367+0000", 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3940000000000001e-06 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719378+0000", 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1204e-05 2024-04-24T18:08:10.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772830+0000", 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053451920999999999 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772896+0000", 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5623000000000005e-05 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772982+0000", 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5729000000000005e-05 2024-04-24T18:08:10.779 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32327 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3389440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.004902512000001, 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053038750000000003, 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32327 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720370+0000", 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720373+0000", 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5830000000000001e-06 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720380+0000", 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1550000000000001e-06 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720384+0000", 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0260000000000001e-06 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773322+0000", 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052937495000000001 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773364+0000", 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2561999999999997e-05 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773409+0000", 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5333000000000001e-05 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32329 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3397632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.004884813, 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053169842000000002, 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32329 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720391+0000", 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7779999999999999e-06 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720396+0000", 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6729999999999997e-06 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720400+0000", 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7179999999999998e-06 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773435+0000", 2024-04-24T18:08:10.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053035169 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773497+0000", 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1715000000000001e-05 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773558+0000", 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1171000000000001e-05 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.788 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32331 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3416064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.788 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:10.788 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.004868684000002, 2024-04-24T18:08:10.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053284320000000003, 2024-04-24T18:08:10.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32331 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720404+0000", 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720406+0000", 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3190000000000002e-06 2024-04-24T18:08:10.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720412+0000", 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8359999999999997e-06 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720416+0000", 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000004e-06 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773583+0000", 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053167076000000001 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773638+0000", 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105999999999997e-05 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773689+0000", 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0521e-05 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32333 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3424256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.004851072000001, 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053395479000000003, 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32333 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720424+0000", 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720430+0000", 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720434+0000", 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8210000000000003e-06 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773721+0000", 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053287297999999997 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773762+0000", 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1245000000000002e-05 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773818+0000", 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5585999999999999e-05 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32388 4.3 4:c61fab1f:::rbd_object_map.117432c990a0.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.002748842999999, 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.055543581000000002, 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:10.797 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32388 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722527+0000", 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0319999999999999e-06 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722535+0000", 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9170000000000006e-06 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722539+0000", 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9709999999999998e-06 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774649+0000", 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052110109000000002 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774719+0000", 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9562999999999998e-05 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775050+0000", 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000331078 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777149+0000", 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020996639999999998 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:10.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777996+0000", 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084650899999999996 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778027+0000", 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0818999999999997e-05 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778068+0000", 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1338000000000003e-05 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.107473857, 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:10.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:08:10.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:08:10.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:10.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:10.808 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:10.809 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:10.726+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:10.809 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:10.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:10.810 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:10.810 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone19 from 0xa760564 (overlap 0x904e81) to 0x2b4ac0 2024-04-24T18:08:10.811 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:10.811 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:10.815 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:10.823+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:10.820 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:10.822 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:10.887 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #18, image image_client.0-clone18 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent19 2024-04-24T18:08:10.916 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:10.924+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:10.967 INFO:teuthology.orchestra.run.smithi049.stdout:2338 read 0x8423e9f thru 0x8425002 (0x1164 bytes) 2024-04-24T18:08:10.967 INFO:teuthology.orchestra.run.smithi049.stdout:2339 read 0x1c5230a thru 0x1c5705b (0x4d52 bytes) 2024-04-24T18:08:10.972 INFO:teuthology.orchestra.run.smithi049.stdout:2340 punch from 0x9ec4e49 to 0x9ec5cb1, (0xe68 bytes) 2024-04-24T18:08:10.975 INFO:teuthology.orchestra.run.smithi049.stdout:2341 punch from 0x5a52444 to 0x5a56811, (0x43cd bytes) 2024-04-24T18:08:10.976 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:10.976 INFO:teuthology.orchestra.run.smithi049.stdout:2343 read 0x1afb5d3 thru 0x1b0a2e5 (0xed13 bytes) 2024-04-24T18:08:10.984 INFO:teuthology.orchestra.run.smithi049.stdout:2344 writesame 0x52f3416 thru 0x52f359e data_size 0x3(0x189 bytes) 2024-04-24T18:08:10.989 INFO:teuthology.orchestra.run.smithi049.stdout:2346 read 0x136293a thru 0x13721dc (0xf8a3 bytes) 2024-04-24T18:08:10.992 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:10.992 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:10.992 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:10.992 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:11.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:11.014 INFO:teuthology.orchestra.run.smithi049.stdout:2348 trunc from 0xa760564 to 0x2829e82 2024-04-24T18:08:11.017 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:11.024+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:11.064 INFO:teuthology.orchestra.run.smithi049.stdout:2349 trunc from 0x2829e82 to 0xcdb0570 2024-04-24T18:08:11.074 INFO:teuthology.orchestra.run.smithi049.stdout:2350 trunc from 0xcdb0570 to 0x783f7f4 2024-04-24T18:08:11.079 INFO:teuthology.orchestra.run.smithi049.stdout:2351 read 0x63c37c7 thru 0x63cab67 (0x73a1 bytes) 2024-04-24T18:08:11.080 INFO:teuthology.orchestra.run.smithi049.stdout:2352 read 0x57e6f32 thru 0x57e79dc (0xaab bytes) 2024-04-24T18:08:11.080 INFO:teuthology.orchestra.run.smithi049.stdout:2353 read 0x2f88f9 thru 0x2f9b42 (0x124a bytes) 2024-04-24T18:08:11.080 INFO:teuthology.orchestra.run.smithi049.stdout:2354 read 0xf05207 thru 0xf09910 (0x470a bytes) 2024-04-24T18:08:11.080 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.081 INFO:teuthology.orchestra.run.smithi049.stdout:2358 writesame 0x6911500 thru 0x691bc36 data_size 0x13(0xa737 bytes) 2024-04-24T18:08:11.088 INFO:teuthology.orchestra.run.smithi049.stdout:2360 punch from 0x652d9da to 0x6530b04, (0x312a bytes) 2024-04-24T18:08:11.088 INFO:teuthology.orchestra.run.smithi049.stdout:2361 read 0x31d90ad thru 0x31e4e59 (0xbdad bytes) 2024-04-24T18:08:11.088 INFO:teuthology.orchestra.run.smithi049.stdout:2363 punch from 0x579affe to 0x579e64e, (0x3650 bytes) 2024-04-24T18:08:11.099 INFO:teuthology.orchestra.run.smithi049.stdout:2364 write 0xc47fb63 thru 0xc48337d (0x381b bytes) 2024-04-24T18:08:11.110 INFO:teuthology.orchestra.run.smithi049.stdout:2365 trunc from 0xc48337e to 0xab92b42 2024-04-24T18:08:11.117 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:11.126+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:11.120 INFO:teuthology.orchestra.run.smithi049.stdout:2366 punch from 0x3ba8acd to 0x3baa059, (0x158c bytes) 2024-04-24T18:08:11.120 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.120 INFO:teuthology.orchestra.run.smithi049.stdout:2368 writesame 0x222301d thru 0x222fdb6 data_size 0x1(0xcd9a bytes) 2024-04-24T18:08:11.138 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.139 INFO:teuthology.orchestra.run.smithi049.stdout:2370 read 0x4837b23 thru 0x4842393 (0xa871 bytes) 2024-04-24T18:08:11.139 INFO:teuthology.orchestra.run.smithi049.stdout:2371 read 0x63c5f3a thru 0x63c616d (0x234 bytes) 2024-04-24T18:08:11.139 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.139 INFO:teuthology.orchestra.run.smithi049.stdout:2373 read 0x7978b1a thru 0x797b955 (0x2e3c bytes) 2024-04-24T18:08:11.139 INFO:teuthology.orchestra.run.smithi049.stdout:2375 punch from 0x4ed1f1e to 0x4edb9d6, (0x9ab8 bytes) 2024-04-24T18:08:11.145 INFO:teuthology.orchestra.run.smithi049.stdout:2377 trunc from 0xab92b42 to 0x49dae6a 2024-04-24T18:08:11.155 INFO:teuthology.orchestra.run.smithi049.stdout:2378 punch from 0x4162945 to 0x416d181, (0xa83c bytes) 2024-04-24T18:08:11.166 INFO:teuthology.orchestra.run.smithi049.stdout:2379 write 0x8dc3b6f thru 0x8dc773a (0x3bcc bytes) 2024-04-24T18:08:11.168 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:11.168 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:11.168 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:11.168 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:11.168 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:11.173 INFO:teuthology.orchestra.run.smithi049.stdout:2380 trunc from 0x8dc773b to 0x9c9cfb2 2024-04-24T18:08:11.178 INFO:teuthology.orchestra.run.smithi049.stdout:2382 writesame 0x75b42ce thru 0x75be18e data_size 0x45(0x9ec1 bytes) 2024-04-24T18:08:11.182 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:11.183 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.183 INFO:teuthology.orchestra.run.smithi049.stdout:2387 read 0x1907800 thru 0x19151ee (0xd9ef bytes) 2024-04-24T18:08:11.191 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.195 INFO:teuthology.orchestra.run.smithi049.stdout:2389 trunc from 0x9c9cfb2 to 0xd119436 2024-04-24T18:08:11.200 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.200 INFO:teuthology.orchestra.run.smithi049.stdout:2392 punch from 0x38c1a0 to 0x392a71, (0x68d1 bytes) 2024-04-24T18:08:11.200 INFO:teuthology.orchestra.run.smithi049.stdout:2393 writesame 0xab446f8 thru 0xab486ed data_size 0x3(0x3ff6 bytes) 2024-04-24T18:08:11.206 INFO:teuthology.orchestra.run.smithi049.stdout:2394 read 0x5c30fea thru 0x5c3a68b (0x96a2 bytes) 2024-04-24T18:08:11.206 INFO:teuthology.orchestra.run.smithi049.stdout:2395 read 0x99fb921 thru 0x9a00f29 (0x5609 bytes) 2024-04-24T18:08:11.206 INFO:teuthology.orchestra.run.smithi049.stdout:2396 read 0x979e5b9 thru 0x97a47d6 (0x621e bytes) 2024-04-24T18:08:11.206 INFO:teuthology.orchestra.run.smithi049.stdout:2398 writesame 0x21f843d thru 0x22073fa data_size 0x35(0xefbe bytes) 2024-04-24T18:08:11.219 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:11.226+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:11.228 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.229 INFO:teuthology.orchestra.run.smithi049.stdout:2400 write 0x29f24ff thru 0x29f8ed6 (0x69d8 bytes) 2024-04-24T18:08:11.234 INFO:teuthology.orchestra.run.smithi049.stdout:2401 read 0x11bbcd8 thru 0x11caa87 (0xedb0 bytes) 2024-04-24T18:08:11.234 INFO:teuthology.orchestra.run.smithi049.stdout:2402 read 0x30b63eb thru 0x30bf0bf (0x8cd5 bytes) 2024-04-24T18:08:11.234 INFO:teuthology.orchestra.run.smithi049.stdout:2403 write 0x101143b thru 0x1012c23 (0x17e9 bytes) 2024-04-24T18:08:11.293 INFO:teuthology.orchestra.run.smithi049.stdout:2404 trunc from 0xd119436 to 0x16e609b 2024-04-24T18:08:11.325 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:11.332+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:11.349 INFO:teuthology.orchestra.run.smithi049.stdout:2405 punch from 0xd8e3dd to 0xd91f1d, (0x3b40 bytes) 2024-04-24T18:08:11.349 INFO:teuthology.orchestra.run.smithi049.stdout:2406 read 0xb68927 thru 0xb75ef7 (0xd5d1 bytes) 2024-04-24T18:08:11.368 INFO:teuthology.orchestra.run.smithi049.stdout:2407 write 0xc84d0ef thru 0xc85565b (0x856d bytes) 2024-04-24T18:08:11.369 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:11.369 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:11.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:11.369 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:11.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40940 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668857643999999, 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036446457000000002, 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40940 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707548+0000", 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.251e-06 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707551+0000", 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0019999999999998e-06 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707554+0000", 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743860+0000", 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036306756000000003 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743915+0000", 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4423999999999999e-05 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743994+0000", 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9207000000000002e-05 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40941 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1490944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668824093, 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036634427999999997, 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40941 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707583+0000", 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2089999999999999e-06 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707587+0000", 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1670000000000001e-06 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707590+0000", 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.554e-06 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744033+0000", 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036443542000000002 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744112+0000", 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8578999999999994e-05 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744215+0000", 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103607 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.377 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40942 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1523712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668813047, 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036756906999999998, 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40942 2024-04-24T18:08:11.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:11.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707594+0000", 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7090000000000001e-06 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707596+0000", 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707598+0000", 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9149999999999999e-06 2024-04-24T18:08:11.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744245+0000", 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036646570000000003 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744295+0000", 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0343000000000002e-05 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744349+0000", 2024-04-24T18:08:11.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3661e-05 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40943 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668804368, 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036956462000000002, 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40943 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707600+0000", 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707602+0000", 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.215e-06 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707605+0000", 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0429999999999999e-06 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707607+0000", 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-06 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744380+0000", 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036773477999999998 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744464+0000", 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4048000000000006e-05 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744557+0000", 2024-04-24T18:08:11.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3043999999999999e-05 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40944 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1589248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668796368000001, 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037054089999999998, 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.386 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40944 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707610+0000", 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1319999999999999e-06 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707613+0000", 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707614+0000", 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8819999999999999e-06 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744569+0000", 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036954539000000002 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744613+0000", 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3671999999999998e-05 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744663+0000", 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0145000000000002e-05 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40945 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668788468000001, 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037248315999999997, 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.390 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40945 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707616+0000", 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707620+0000", 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1999999999999999e-06 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707623+0000", 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1669999999999999e-06 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707625+0000", 2024-04-24T18:08:11.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1110000000000002e-06 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744693+0000", 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037068564999999998 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744773+0000", 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9172999999999996e-05 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744865+0000", 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2305000000000005e-05 2024-04-24T18:08:11.393 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40946 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668777925000001, 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037340299, 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40946 2024-04-24T18:08:11.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:11.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707628+0000", 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2699999999999999e-06 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707631+0000", 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9519999999999999e-06 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707633+0000", 2024-04-24T18:08:11.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8619999999999999e-06 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744876+0000", 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037242550999999999 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744918+0000", 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2546000000000002e-05 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744967+0000", 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9320999999999998e-05 2024-04-24T18:08:11.397 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40947 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668769869, 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037537860999999999, 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40947 2024-04-24T18:08:11.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707636+0000", 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.186e-06 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707639+0000", 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707641+0000", 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.86e-06 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744997+0000", 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037356364000000003 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745076+0000", 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8355000000000005e-05 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745173+0000", 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7379000000000004e-05 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.401 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40948 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1720320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668761945, 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037655182000000002, 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40948 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707644+0000", 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.269e-06 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707647+0000", 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9359999999999999e-06 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707649+0000", 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.815e-06 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745200+0000", 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037551189999999998 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745246+0000", 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6214999999999997e-05 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745298+0000", 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1960999999999999e-05 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.405 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40949 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668753916, 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037846454000000002, 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40949 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:11.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707652+0000", 2024-04-24T18:08:11.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.13e-06 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707655+0000", 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707658+0000", 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5770000000000001e-06 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745329+0000", 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037671351999999998 2024-04-24T18:08:11.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745408+0000", 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745498+0000", 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0043000000000005e-05 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40950 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.409 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668745235999999, 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037946947000000002, 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40950 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707661+0000", 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.15e-06 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707664+0000", 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.965e-06 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707666+0000", 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745508+0000", 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037841977999999998 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745555+0000", 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7098999999999997e-05 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745607+0000", 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1591999999999997e-05 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40951 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.413 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668736755999999, 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038137931999999999, 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40951 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:11.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707670+0000", 2024-04-24T18:08:11.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7010000000000001e-06 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707673+0000", 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9119999999999998e-06 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707675+0000", 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9429999999999999e-06 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745637+0000", 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037962379999999997 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745715+0000", 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7488000000000001e-05 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745806+0000", 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1723000000000003e-05 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40952 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:11.417 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.66872813, 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038236762000000001, 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40952 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:11.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707678+0000", 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.093e-06 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707681+0000", 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707683+0000", 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.951e-06 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745817+0000", 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038133854000000002 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745862+0000", 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5027000000000001e-05 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745914+0000", 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2120999999999998e-05 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.421 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40953 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668720097, 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038424458000000002, 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40953 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.422 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707686+0000", 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.567e-06 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707689+0000", 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9220000000000001e-06 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707691+0000", 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9369999999999998e-06 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745943+0000", 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038252047999999997 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746018+0000", 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4938000000000007e-05 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746109+0000", 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1255000000000006e-05 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.425 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40954 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1916928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668711588000001, 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038528139000000003, 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40954 2024-04-24T18:08:11.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707693+0000", 2024-04-24T18:08:11.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.065e-06 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707697+0000", 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.881e-06 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707699+0000", 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.88e-06 2024-04-24T18:08:11.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746120+0000", 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038420937000000002 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746171+0000", 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0555e-05 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746222+0000", 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1025e-05 2024-04-24T18:08:11.429 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40955 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668703768, 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718855000000003, 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.430 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40955 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707702+0000", 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.212e-06 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707706+0000", 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.534e-06 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707708+0000", 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746254+0000", 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038545870000000003 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746331+0000", 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7805000000000005e-05 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746420+0000", 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8721000000000007e-05 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40956 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.66869515, 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038823347000000001, 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40956 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:11.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707711+0000", 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1659999999999999e-06 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707714+0000", 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.937e-06 2024-04-24T18:08:11.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707716+0000", 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9920000000000002e-06 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746435+0000", 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718869000000003 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746482+0000", 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6882999999999998e-05 2024-04-24T18:08:11.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746533+0000", 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1711e-05 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40957 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668686876000001, 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039015052000000001, 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40957 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707721+0000", 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3110000000000001e-06 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707725+0000", 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2669999999999998e-06 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707727+0000", 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746563+0000", 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038836751000000003 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746641+0000", 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7900999999999998e-05 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746733+0000", 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2015999999999994e-05 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40958 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2048000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668676183000001, 2024-04-24T18:08:11.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039110399999999997, 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40958 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707730+0000", 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2640000000000001e-06 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707733+0000", 2024-04-24T18:08:11.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9170000000000002e-06 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707735+0000", 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746744+0000", 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039008958000000003 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746789+0000", 2024-04-24T18:08:11.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5019999999999999e-05 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746839+0000", 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0531999999999997e-05 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40959 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.668668009999999, 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039290798000000002, 2024-04-24T18:08:11.446 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40959 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707738+0000", 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.24e-06 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707742+0000", 2024-04-24T18:08:11.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6200000000000001e-06 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707744+0000", 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.615e-06 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746859+0000", 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039115095000000003 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746938+0000", 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:11.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.747028+0000", 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9742999999999998e-05 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.450 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:11.451 INFO:teuthology.orchestra.run.smithi049.stdout:2410 trunc from 0xc85565c to 0xc20edfd 2024-04-24T18:08:11.452 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:11.453 INFO:teuthology.orchestra.run.smithi049.stdout:2412 trunc from 0xc20edfd to 0x31d82dc 2024-04-24T18:08:11.453 INFO:teuthology.orchestra.run.smithi049.stdout:2413 writesame 0xa3566dd thru 0xa359120 data_size 0xa(0x2a44 bytes) 2024-04-24T18:08:11.453 INFO:teuthology.orchestra.run.smithi049.stdout:2415 write 0x13c1d68 thru 0x13c3b60 (0x1df9 bytes) 2024-04-24T18:08:11.453 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:11.459+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:11.464 INFO:teuthology.orchestra.run.smithi049.stdout:2416 read 0x2628739 thru 0x262fdf1 (0x76b9 bytes) 2024-04-24T18:08:11.464 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.464 INFO:teuthology.orchestra.run.smithi049.stdout:2418 write 0xb730d5 thru 0xb7f115 (0xc041 bytes) 2024-04-24T18:08:11.552 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:11.560+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:11.594 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:11.595 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:11.595 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:11.595 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:11.605 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:11.653 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:11.661+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:11.683 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.6 to 1 (10000) 2024-04-24T18:08:11.690 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:08:11.698 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-24T18:08:11.722 INFO:teuthology.orchestra.run.smithi049.stdout:2421 read 0x271e27d thru 0x272177e (0x3502 bytes) 2024-04-24T18:08:11.722 INFO:teuthology.orchestra.run.smithi049.stdout:2422 write 0x548c324 thru 0x5494ddb (0x8ab8 bytes) 2024-04-24T18:08:11.737 INFO:teuthology.orchestra.run.smithi049.stdout:2423 write 0xa621fd1 thru 0xa624a7c (0x2aac bytes) 2024-04-24T18:08:11.750 INFO:teuthology.orchestra.run.smithi049.stdout:2424 write 0xacae89c thru 0xacb1763 (0x2ec8 bytes) 2024-04-24T18:08:11.755 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:11.761+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:11.757 INFO:teuthology.orchestra.run.smithi049.stdout:2425 write 0x721adc4 thru 0x7221975 (0x6bb2 bytes) 2024-04-24T18:08:11.764 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:11.764 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:11.764 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:11.764 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:11.764 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:11.765 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.765 INFO:teuthology.orchestra.run.smithi049.stdout:2428 write 0x334826a thru 0x334dc48 (0x59df bytes) 2024-04-24T18:08:11.772 INFO:teuthology.orchestra.run.smithi049.stdout:2429 write 0x821d13d thru 0x822ba4a (0xe90e bytes) 2024-04-24T18:08:11.776 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:11.779 INFO:teuthology.orchestra.run.smithi049.stdout:2430 writesame 0x8e1c646 thru 0x8e20704 data_size 0x5(0x40bf bytes) 2024-04-24T18:08:11.786 INFO:teuthology.orchestra.run.smithi049.stdout:2431 write 0x37ce690 thru 0x37d1f7e (0x38ef bytes) 2024-04-24T18:08:11.796 INFO:teuthology.orchestra.run.smithi049.stdout:2432 punch from 0x1b68f44 to 0x1b785b8, (0xf674 bytes) 2024-04-24T18:08:11.796 INFO:teuthology.orchestra.run.smithi049.stdout:2433 punch from 0x7d33ef7 to 0x7d38e87, (0x4f90 bytes) 2024-04-24T18:08:11.796 INFO:teuthology.orchestra.run.smithi049.stdout:2434 punch from 0x3ebee5d to 0x3ecc66e, (0xd811 bytes) 2024-04-24T18:08:11.797 INFO:teuthology.orchestra.run.smithi049.stdout:2436 write 0x363875e thru 0x364818b (0xfa2e bytes) 2024-04-24T18:08:11.803 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:11.803 INFO:teuthology.orchestra.run.smithi049.stdout:2439 write 0x14ec71d thru 0x14eed73 (0x2657 bytes) 2024-04-24T18:08:11.855 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:11.863+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:11.868 INFO:teuthology.orchestra.run.smithi049.stdout:2440 read 0x94734c7 thru 0x94756c0 (0x21fa bytes) 2024-04-24T18:08:11.881 INFO:teuthology.orchestra.run.smithi049.stdout:2441 trunc from 0xacb1764 to 0x38d16e 2024-04-24T18:08:11.957 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:11.965+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57060 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 337920~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.920378+0000", 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.068732013999999994, 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048587198999999998, 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57060 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920378+0000", 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920378+0000", 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920378+0000", 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920380+0000", 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7190000000000001e-06 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920386+0000", 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9290000000000003e-06 2024-04-24T18:08:11.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920389+0000", 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2310000000000001e-06 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.968788+0000", 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048398398000000002 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.968877+0000", 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9061000000000007e-05 2024-04-24T18:08:11.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.968965+0000", 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8189000000000006e-05 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57062 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 342016~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.920393+0000", 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.068717064999999994, 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049229848999999999, 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57062 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920393+0000", 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920393+0000", 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920393+0000", 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920394+0000", 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8759999999999999e-06 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920400+0000", 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2220000000000001e-06 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920403+0000", 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9380000000000001e-06 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.968994+0000", 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048591571 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.969080+0000", 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5937000000000001e-05 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.969623+0000", 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00054262099999999997 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57063 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 346112~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.920408+0000", 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.068701518000000003, 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049462138000000003, 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57063 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920408+0000", 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920408+0000", 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920408+0000", 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920410+0000", 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1689999999999999e-06 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920416+0000", 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4960000000000004e-06 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920420+0000", 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1880000000000004e-06 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.969689+0000", 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049269230999999997 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.969778+0000", 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8769000000000003e-05 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.969871+0000", 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2575e-05 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57065 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 350208~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.920424+0000", 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.068686074999999999, 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049666767000000001, 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.995 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57065 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920424+0000", 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920424+0000", 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920423+0000", 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920426+0000", 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3980000000000002e-06 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920431+0000", 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6339999999999998e-06 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920435+0000", 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.456e-06 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.969913+0000", 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049477858 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.969996+0000", 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2960000000000005e-05 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.970091+0000", 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4846999999999995e-05 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57067 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 354304~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.920439+0000", 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.068670863999999998, 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049860539000000002, 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:11.999 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57067 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920439+0000", 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920439+0000", 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920439+0000", 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920441+0000", 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2560000000000001e-06 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920446+0000", 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1490000000000002e-06 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920449+0000", 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.021e-06 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.970120+0000", 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049671327000000001 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.970207+0000", 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6661000000000003e-05 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.970300+0000", 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2451e-05 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57068 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 358400~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.920453+0000", 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.068656574999999997, 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050035218999999999, 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.003 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57068 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920453+0000", 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920453+0000", 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920453+0000", 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920455+0000", 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.086e-06 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920461+0000", 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7710000000000003e-06 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.920464+0000", 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4690000000000002e-06 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.970328+0000", 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.049863745000000001 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.970406+0000", 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7945e-05 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.970489+0000", 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2569000000000002e-05 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57289 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925662+0000", 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063447719, 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048506932000000003, 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.007 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57289 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925662+0000", 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925662+0000", 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925662+0000", 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925664+0000", 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7030000000000002e-06 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925671+0000", 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4130000000000003e-06 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925675+0000", 2024-04-24T18:08:12.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7670000000000001e-06 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974162+0000", 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048487800999999997 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974169+0000", 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7959999999999998e-06 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57290 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925679+0000", 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063431181000000003, 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048519305999999998, 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57290 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925679+0000", 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925679+0000", 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925678+0000", 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925681+0000", 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1220000000000002e-06 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925686+0000", 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6219999999999997e-06 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925695+0000", 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3510000000000008e-06 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974192+0000", 2024-04-24T18:08:12.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048496145999999997 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974198+0000", 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.409e-06 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57291 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925702+0000", 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063408013999999999, 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048524288999999998, 2024-04-24T18:08:12.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57291 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925702+0000", 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925702+0000", 2024-04-24T18:08:12.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925702+0000", 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925714+0000", 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2557e-05 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925721+0000", 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5350000000000001e-06 2024-04-24T18:08:12.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925724+0000", 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8700000000000002e-06 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974219+0000", 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048494163999999999 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974226+0000", 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5499999999999997e-06 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.017 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57292 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925729+0000", 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063380862999999996, 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048526527, 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57292 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.018 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925729+0000", 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925729+0000", 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925729+0000", 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925731+0000", 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1830000000000001e-06 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925737+0000", 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7599999999999999e-06 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925740+0000", 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.608e-06 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974249+0000", 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048509047 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974256+0000", 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3289999999999999e-06 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57293 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.021 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925744+0000", 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063365475000000004, 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048548984000000003, 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57293 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925744+0000", 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925744+0000", 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925744+0000", 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925746+0000", 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0420000000000001e-06 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925752+0000", 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7599999999999999e-06 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925755+0000", 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4359999999999998e-06 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974287+0000", 2024-04-24T18:08:12.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048531257000000001 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974293+0000", 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8379999999999996e-06 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57294 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925759+0000", 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063350504000000002, 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048562503999999999, 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57294 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925759+0000", 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925759+0000", 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925759+0000", 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925766+0000", 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3719999999999999e-06 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925772+0000", 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7209999999999999e-06 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925776+0000", 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8489999999999999e-06 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974316+0000", 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048539634999999998 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974322+0000", 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2689999999999996e-06 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57295 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925780+0000", 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063329604999999997, 2024-04-24T18:08:12.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048577624999999999, 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57295 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925780+0000", 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925780+0000", 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925780+0000", 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925782+0000", 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.277e-06 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925788+0000", 2024-04-24T18:08:12.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.003e-06 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925792+0000", 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4460000000000001e-06 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974352+0000", 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048560218000000002 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974358+0000", 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.049e-06 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57296 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925796+0000", 2024-04-24T18:08:12.034 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063313876000000005, 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048590492999999998, 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57296 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925796+0000", 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925796+0000", 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925796+0000", 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925798+0000", 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3989999999999998e-06 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925804+0000", 2024-04-24T18:08:12.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0669999999999997e-06 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925808+0000", 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6459999999999999e-06 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974380+0000", 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048572652000000001 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974387+0000", 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1160000000000004e-06 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57297 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925812+0000", 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063297786999999994, 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048603802000000002, 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57297 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925812+0000", 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925812+0000", 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925812+0000", 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925814+0000", 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925820+0000", 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2729999999999999e-06 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925824+0000", 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.675e-06 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974410+0000", 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048585432999999997 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974416+0000", 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4189999999999999e-06 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.042 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57298 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925828+0000", 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063281688000000003, 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048620438000000002, 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57298 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925828+0000", 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925828+0000", 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925828+0000", 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925830+0000", 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.153e-06 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925836+0000", 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.93e-06 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925839+0000", 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.433e-06 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974438+0000", 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048598293000000001 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974449+0000", 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0969e-05 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57299 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925843+0000", 2024-04-24T18:08:12.046 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063266428999999999, 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048633453, 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57299 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925843+0000", 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925843+0000", 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925843+0000", 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925845+0000", 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0729999999999999e-06 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925851+0000", 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6130000000000003e-06 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925867+0000", 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5860000000000001e-05 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974471+0000", 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048604033999999997 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974477+0000", 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2230000000000004e-06 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57300 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925872+0000", 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063238206000000005, 2024-04-24T18:08:12.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048633141999999997, 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57300 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925872+0000", 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925872+0000", 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925871+0000", 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925874+0000", 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.509e-06 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925880+0000", 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3389999999999998e-06 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925884+0000", 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5269999999999999e-06 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974499+0000", 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048614906999999999 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974505+0000", 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2589999999999997e-06 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57301 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925893+0000", 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063217368999999995, 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048640297999999998, 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.054 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57301 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925893+0000", 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925893+0000", 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925892+0000", 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925895+0000", 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3290000000000001e-06 2024-04-24T18:08:12.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925901+0000", 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1630000000000001e-06 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925904+0000", 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6780000000000002e-06 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974525+0000", 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.048620904999999999 2024-04-24T18:08:12.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.974533+0000", 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5800000000000003e-06 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:57302 4.1c 4.c5b6293c (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:11.925909+0000", 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.063201370000000007, 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050297627999999997, 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.058 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 57302 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925909+0000", 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925909+0000", 2024-04-24T18:08:12.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925908+0000", 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925910+0000", 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1720000000000001e-06 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925916+0000", 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8159999999999999e-06 2024-04-24T18:08:12.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.925920+0000", 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.976197+0000", 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050277395000000003 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:11.976206+0000", 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1439999999999992e-06 2024-04-24T18:08:12.061 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.062 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.062 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.062 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.062 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.062 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:12.063 INFO:teuthology.orchestra.run.smithi049.stdout:2442 read 0x37eafe thru 0x388a2e (0x9f31 bytes) 2024-04-24T18:08:12.063 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:12.071+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:12.064 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2443 write 0x65befc6 thru 0x65cdb7c (0xebb7 bytes) 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2444 read 0x4e8e5a7 thru 0x4e9c795 (0xe1ef bytes) 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2445 writesame 0x958cae0 thru 0x9591eea data_size 0xd(0x540b bytes) 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2448 trunc from 0x9591eeb to 0x6d674e4 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2450 trunc from 0x6d674e4 to 0x5f2fc6f 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2451 write 0x5c19f7c thru 0x5c23127 (0x91ac bytes) 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2452 read 0x3c4cd9c thru 0x3c56c5b (0x9ec0 bytes) 2024-04-24T18:08:12.065 INFO:teuthology.orchestra.run.smithi049.stdout:2453 write 0xb4000a5 thru 0xb402c12 (0x2b6e bytes) 2024-04-24T18:08:12.067 INFO:teuthology.orchestra.run.smithi049.stdout:2454 read 0x85f950b thru 0x85ff186 (0x5c7c bytes) 2024-04-24T18:08:12.067 INFO:teuthology.orchestra.run.smithi049.stdout:2455 write 0xa8979cc thru 0xa899770 (0x1da5 bytes) 2024-04-24T18:08:12.076 INFO:teuthology.orchestra.run.smithi049.stdout:2456 punch from 0xa740f39 to 0xa746384, (0x544b bytes) 2024-04-24T18:08:12.076 INFO:teuthology.orchestra.run.smithi049.stdout:2457 read 0x7c563da thru 0x7c632d2 (0xcef9 bytes) 2024-04-24T18:08:12.084 INFO:teuthology.orchestra.run.smithi049.stdout:2458 trunc from 0xb402c13 to 0x4725b27 2024-04-24T18:08:12.096 INFO:teuthology.orchestra.run.smithi049.stdout:2459 write 0x925a12 thru 0x92bc54 (0x6243 bytes) 2024-04-24T18:08:12.104 INFO:teuthology.orchestra.run.smithi049.stdout:2460 trunc from 0x4725b27 to 0x47fe2e8 2024-04-24T18:08:12.109 INFO:teuthology.orchestra.run.smithi049.stdout:2461 flatten 2024-04-24T18:08:12.170 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:12.175+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:12.266 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:12.273+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:12.301 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:12.301 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59750 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.250163+0000", 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.042473415, 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.042492936000000002, 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59750 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.302 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250163+0000", 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250163+0000", 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.248378+0000", 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9982152 2024-04-24T18:08:12.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250168+0000", 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001789452 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250181+0000", 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.345e-05 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250187+0000", 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3589999999999996e-06 2024-04-24T18:08:12.304 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59834 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.254252+0000", 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.038383739, 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038447522999999997, 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.305 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59834 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254252+0000", 2024-04-24T18:08:12.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254252+0000", 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254252+0000", 2024-04-24T18:08:12.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254255+0000", 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6910000000000002e-06 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254262+0000", 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3339999999999996e-06 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254266+0000", 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3390000000000003e-06 2024-04-24T18:08:12.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59930 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260414+0000", 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.032222491999999998, 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.032345684999999999, 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59930 2024-04-24T18:08:12.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260414+0000", 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260414+0000", 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260413+0000", 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260416+0000", 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2299999999999998e-06 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260421+0000", 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2480000000000004e-06 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260424+0000", 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2830000000000002e-06 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60059 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268541+0000", 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024094877000000001, 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.024261947999999998, 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60059 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268541+0000", 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268541+0000", 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268541+0000", 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268543+0000", 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4430000000000002e-06 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268549+0000", 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9320000000000001e-06 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268553+0000", 2024-04-24T18:08:12.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6009999999999999e-06 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60090 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274206+0000", 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018430102, 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018641174, 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60090 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274206+0000", 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274206+0000", 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274206+0000", 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274207+0000", 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.404e-06 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274210+0000", 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9299999999999999e-06 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274212+0000", 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2340000000000001e-06 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60188 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280838+0000", 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011798568000000001, 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012045828, 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60188 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280838+0000", 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280838+0000", 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280837+0000", 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280839+0000", 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2800000000000002e-06 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280845+0000", 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7389999999999996e-06 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280849+0000", 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8179999999999997e-06 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.323 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60316 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287721+0000", 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0049146909999999997, 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051942739999999996, 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60316 2024-04-24T18:08:12.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287721+0000", 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287721+0000", 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287721+0000", 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287723+0000", 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.438e-06 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287726+0000", 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6430000000000001e-06 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287729+0000", 2024-04-24T18:08:12.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2579999999999998e-06 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60380 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290702+0000", 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00193366, 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0022635099999999998, 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60380 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290702+0000", 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290702+0000", 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290702+0000", 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290705+0000", 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5900000000000002e-06 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290711+0000", 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.657e-06 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290715+0000", 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9879999999999998e-06 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59752 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.250194+0000", 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.042441786000000002, 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.042806856999999997, 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.330 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59752 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250194+0000", 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250194+0000", 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250194+0000", 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250197+0000", 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.847e-06 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250203+0000", 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.443e-06 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250207+0000", 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8879999999999999e-06 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59835 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.254271+0000", 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.038364769, 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038765189999999998, 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59835 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.334 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254271+0000", 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254271+0000", 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254271+0000", 2024-04-24T18:08:12.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254274+0000", 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254280+0000", 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1369999999999999e-06 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254284+0000", 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8890000000000004e-06 2024-04-24T18:08:12.336 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59931 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260648+0000", 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031988368000000003, 2024-04-24T18:08:12.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.032429798000000003, 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59931 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.338 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260648+0000", 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260648+0000", 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260647+0000", 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260651+0000", 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0489999999999997e-06 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260659+0000", 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0970000000000006e-06 2024-04-24T18:08:12.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260667+0000", 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2420000000000003e-06 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60060 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268557+0000", 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024079064000000001, 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.024556761, 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60060 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268557+0000", 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268557+0000", 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268557+0000", 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268559+0000", 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3920000000000001e-06 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268565+0000", 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268569+0000", 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.839e-06 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60091 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274214+0000", 2024-04-24T18:08:12.346 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018421752, 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018932628, 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60091 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274214+0000", 2024-04-24T18:08:12.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274214+0000", 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274214+0000", 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274216+0000", 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.42e-06 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274218+0000", 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8959999999999999e-06 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274221+0000", 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0640000000000001e-06 2024-04-24T18:08:12.349 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60189 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282188+0000", 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010448084, 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.01101647, 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.350 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60189 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282188+0000", 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282188+0000", 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280854+0000", 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9986663 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282195+0000", 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00134095 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282207+0000", 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2078e-05 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282214+0000", 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7059999999999998e-06 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60317 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287731+0000", 2024-04-24T18:08:12.353 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0049053839999999996, 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0055098580000000003, 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60317 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287731+0000", 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287731+0000", 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287731+0000", 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287732+0000", 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3939999999999999e-06 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287735+0000", 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287737+0000", 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.007e-06 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60381 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290719+0000", 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0019168309999999999, 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002564153, 2024-04-24T18:08:12.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60381 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290719+0000", 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290719+0000", 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290719+0000", 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290721+0000", 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.328e-06 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290727+0000", 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9630000000000003e-06 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290731+0000", 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.799e-06 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59754 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.250211+0000", 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.042424721999999998, 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.043109051000000002, 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59754 2024-04-24T18:08:12.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250211+0000", 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250211+0000", 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250211+0000", 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250213+0000", 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.277e-06 2024-04-24T18:08:12.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250220+0000", 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250224+0000", 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5379999999999998e-06 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59836 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.254288+0000", 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.038348010000000002, 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039066391999999998, 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59836 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254288+0000", 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254288+0000", 2024-04-24T18:08:12.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254288+0000", 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254290+0000", 2024-04-24T18:08:12.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254296+0000", 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0260000000000004e-06 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254300+0000", 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7560000000000001e-06 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59932 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260672+0000", 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031964148999999997, 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.032748518999999997, 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59932 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260672+0000", 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260672+0000", 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260671+0000", 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260679+0000", 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3549999999999999e-06 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260686+0000", 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0040000000000002e-06 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260690+0000", 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5329999999999996e-06 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.372 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60061 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268573+0000", 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024063173, 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.024932677, 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60061 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268573+0000", 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268573+0000", 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268573+0000", 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268575+0000", 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268581+0000", 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9229999999999999e-06 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268584+0000", 2024-04-24T18:08:12.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.399e-06 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60092 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274223+0000", 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018413637, 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019337222000000001, 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60092 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274223+0000", 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274223+0000", 2024-04-24T18:08:12.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274222+0000", 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274224+0000", 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.308e-06 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274226+0000", 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274228+0000", 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9920000000000002e-06 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60190 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282221+0000", 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010414856, 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011378588, 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60190 2024-04-24T18:08:12.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282221+0000", 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282221+0000", 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282221+0000", 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282224+0000", 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2219999999999998e-06 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282231+0000", 2024-04-24T18:08:12.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0509999999999999e-06 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282236+0000", 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7570000000000004e-06 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60318 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287739+0000", 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0048971199999999996, 2024-04-24T18:08:12.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0059077390000000004, 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60318 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287739+0000", 2024-04-24T18:08:12.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287739+0000", 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287739+0000", 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287740+0000", 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3829999999999999e-06 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287744+0000", 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5829999999999998e-06 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287746+0000", 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.052e-06 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60382 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290735+0000", 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0019009439999999999, 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0029911959999999998, 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60382 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290735+0000", 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290735+0000", 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290735+0000", 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290737+0000", 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4930000000000001e-06 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290743+0000", 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.857e-06 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290748+0000", 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4009999999999996e-06 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59756 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.250236+0000", 2024-04-24T18:08:12.390 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.042400291, 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.043554849, 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59756 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250236+0000", 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250236+0000", 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250235+0000", 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250238+0000", 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250244+0000", 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2580000000000001e-06 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.250248+0000", 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0489999999999997e-06 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.393 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59837 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.254305+0000", 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.038331109000000002, 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039520567, 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59837 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254305+0000", 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254305+0000", 2024-04-24T18:08:12.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254304+0000", 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254307+0000", 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254314+0000", 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6800000000000004e-06 2024-04-24T18:08:12.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254318+0000", 2024-04-24T18:08:12.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8829999999999999e-06 2024-04-24T18:08:12.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.398 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.399 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59933 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.399 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260695+0000", 2024-04-24T18:08:12.399 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031940726000000003, 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.033169351, 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59933 2024-04-24T18:08:12.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.401 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260695+0000", 2024-04-24T18:08:12.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260695+0000", 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260695+0000", 2024-04-24T18:08:12.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260697+0000", 2024-04-24T18:08:12.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.469e-06 2024-04-24T18:08:12.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260703+0000", 2024-04-24T18:08:12.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0179999999999998e-06 2024-04-24T18:08:12.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260708+0000", 2024-04-24T18:08:12.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8969999999999999e-06 2024-04-24T18:08:12.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.405 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.406 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60062 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.406 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268588+0000", 2024-04-24T18:08:12.406 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024047763, 2024-04-24T18:08:12.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.025359484000000002, 2024-04-24T18:08:12.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60062 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268588+0000", 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268588+0000", 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268588+0000", 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268590+0000", 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.243e-06 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268596+0000", 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0249999999999999e-06 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268600+0000", 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.614e-06 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60093 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274231+0000", 2024-04-24T18:08:12.409 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018405597999999999, 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019792067, 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60093 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274231+0000", 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274231+0000", 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274230+0000", 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274232+0000", 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3459999999999999e-06 2024-04-24T18:08:12.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274235+0000", 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8459999999999999e-06 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274237+0000", 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0030000000000001e-06 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.412 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60191 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282242+0000", 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010394547000000001, 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011819069, 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60191 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282242+0000", 2024-04-24T18:08:12.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282242+0000", 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282241+0000", 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282244+0000", 2024-04-24T18:08:12.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8289999999999999e-06 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282250+0000", 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5590000000000002e-06 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282254+0000", 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.014e-06 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60319 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287748+0000", 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0048882939999999996, 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063571360000000002, 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60319 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287748+0000", 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287748+0000", 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287748+0000", 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287749+0000", 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.48e-06 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287752+0000", 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8949999999999998e-06 2024-04-24T18:08:12.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287754+0000", 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9819999999999998e-06 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60383 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290769+0000", 2024-04-24T18:08:12.419 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0018676299999999999, 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.003423125, 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60383 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290769+0000", 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290769+0000", 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290768+0000", 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290771+0000", 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.841e-06 2024-04-24T18:08:12.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290777+0000", 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6630000000000004e-06 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290782+0000", 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3059999999999999e-06 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59838 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.254322+0000", 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.038313765999999999, 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039926730000000001, 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59838 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254322+0000", 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254322+0000", 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254322+0000", 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254325+0000", 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5500000000000001e-06 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254330+0000", 2024-04-24T18:08:12.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6389999999999997e-06 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254334+0000", 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59934 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260713+0000", 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031922969000000002, 2024-04-24T18:08:12.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.033574585999999997, 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59934 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260713+0000", 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260713+0000", 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260713+0000", 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260715+0000", 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4430000000000002e-06 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260721+0000", 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1340000000000001e-06 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260725+0000", 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1069999999999998e-06 2024-04-24T18:08:12.429 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60063 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268604+0000", 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024031874000000002, 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.025730610000000001, 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60063 2024-04-24T18:08:12.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268604+0000", 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268604+0000", 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268604+0000", 2024-04-24T18:08:12.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268606+0000", 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268612+0000", 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8189999999999997e-06 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268616+0000", 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8070000000000001e-06 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60094 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274239+0000", 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018397495999999999, 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020163205999999999, 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60094 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274239+0000", 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274239+0000", 2024-04-24T18:08:12.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274238+0000", 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274243+0000", 2024-04-24T18:08:12.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2100000000000003e-06 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274246+0000", 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.145e-06 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274248+0000", 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0729999999999999e-06 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60192 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282259+0000", 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010377154, 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012183678, 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60192 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.437 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282259+0000", 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282259+0000", 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282259+0000", 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282261+0000", 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282267+0000", 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9970000000000004e-06 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282271+0000", 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60320 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287756+0000", 2024-04-24T18:08:12.440 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0048800750000000002, 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0067218210000000002, 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60320 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287756+0000", 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287756+0000", 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287756+0000", 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287757+0000", 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.432e-06 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287760+0000", 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8830000000000002e-06 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287762+0000", 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9609999999999999e-06 2024-04-24T18:08:12.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60384 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292171+0000", 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00046474899999999998, 2024-04-24T18:08:12.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0023481539999999999, 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60384 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.445 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292171+0000", 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292171+0000", 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290786+0000", 2024-04-24T18:08:12.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9986143 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292175+0000", 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001389693 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292186+0000", 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0421000000000001e-05 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292191+0000", 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3730000000000001e-06 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59839 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.254338+0000", 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.038298196999999999, 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.040217768000000001, 2024-04-24T18:08:12.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59839 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254338+0000", 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254338+0000", 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254338+0000", 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254345+0000", 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1420000000000004e-06 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254351+0000", 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0689999999999998e-06 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254355+0000", 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6799999999999999e-06 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59935 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260731+0000", 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031905498999999997, 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.033859586999999997, 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.452 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59935 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260731+0000", 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260731+0000", 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260730+0000", 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260733+0000", 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4250000000000001e-06 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260739+0000", 2024-04-24T18:08:12.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1369999999999999e-06 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260744+0000", 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4299999999999997e-06 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60064 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268620+0000", 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024016197999999999, 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026004053999999999, 2024-04-24T18:08:12.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60064 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268620+0000", 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268620+0000", 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268620+0000", 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268622+0000", 2024-04-24T18:08:12.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3190000000000002e-06 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268628+0000", 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7250000000000002e-06 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268631+0000", 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.726e-06 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.458 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60095 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276185+0000", 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016450704999999999, 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018474253, 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60095 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276185+0000", 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276185+0000", 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274250+0000", 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9980645 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276190+0000", 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0019396890000000001 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276199+0000", 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0710000000000001e-06 2024-04-24T18:08:12.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276208+0000", 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2070000000000001e-06 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60193 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282276+0000", 2024-04-24T18:08:12.462 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010360553999999999, 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012419327000000001, 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60193 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282276+0000", 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282276+0000", 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282275+0000", 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282278+0000", 2024-04-24T18:08:12.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282284+0000", 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1759999999999998e-06 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282288+0000", 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.109e-06 2024-04-24T18:08:12.465 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60321 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287764+0000", 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0048718800000000003, 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069665719999999999, 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.466 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60321 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287764+0000", 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287764+0000", 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287764+0000", 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287766+0000", 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.607e-06 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287770+0000", 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7400000000000004e-06 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287775+0000", 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7060000000000003e-06 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60385 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292198+0000", 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00043841300000000002, 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.002567223, 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.469 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60385 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292198+0000", 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292198+0000", 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292197+0000", 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292200+0000", 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7369999999999998e-06 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292206+0000", 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8930000000000002e-06 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292210+0000", 2024-04-24T18:08:12.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1989999999999999e-06 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59840 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.257863+0000", 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034773291999999997, 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036947020999999997, 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59840 2024-04-24T18:08:12.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257863+0000", 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257863+0000", 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257862+0000", 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257867+0000", 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6110000000000001e-06 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257880+0000", 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2859e-05 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257887+0000", 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59936 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260749+0000", 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031887221, 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034098333000000002, 2024-04-24T18:08:12.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59936 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260749+0000", 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260749+0000", 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260749+0000", 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260751+0000", 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4219999999999999e-06 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260757+0000", 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0580000000000003e-06 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260761+0000", 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9890000000000003e-06 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.479 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60065 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270165+0000", 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022471628, 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.024724274000000001, 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60065 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270165+0000", 2024-04-24T18:08:12.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270165+0000", 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268635+0000", 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9984703 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270170+0000", 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0015348219999999999 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270187+0000", 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7524999999999999e-05 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270194+0000", 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3550000000000001e-06 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.482 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60096 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276215+0000", 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016421194, 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018706911, 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.483 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60096 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.484 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276215+0000", 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276215+0000", 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276214+0000", 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276218+0000", 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0410000000000002e-06 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276224+0000", 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6719999999999998e-06 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276228+0000", 2024-04-24T18:08:12.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2880000000000003e-06 2024-04-24T18:08:12.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.488 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.489 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60194 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.489 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282293+0000", 2024-04-24T18:08:12.489 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010343609, 2024-04-24T18:08:12.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012666014999999999, 2024-04-24T18:08:12.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.490 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.490 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.490 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.490 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60194 2024-04-24T18:08:12.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.490 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282293+0000", 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282293+0000", 2024-04-24T18:08:12.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282292+0000", 2024-04-24T18:08:12.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282295+0000", 2024-04-24T18:08:12.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4530000000000001e-06 2024-04-24T18:08:12.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282301+0000", 2024-04-24T18:08:12.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0809999999999999e-06 2024-04-24T18:08:12.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282304+0000", 2024-04-24T18:08:12.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60322 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287793+0000", 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0048435759999999996, 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0072012600000000001, 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.495 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60322 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287793+0000", 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287793+0000", 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287792+0000", 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287795+0000", 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9579999999999999e-06 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287802+0000", 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7449999999999998e-06 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287806+0000", 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1880000000000004e-06 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60386 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292215+0000", 2024-04-24T18:08:12.498 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.000421304, 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0028125369999999999, 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60386 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292215+0000", 2024-04-24T18:08:12.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292215+0000", 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292214+0000", 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292217+0000", 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5349999999999999e-06 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292224+0000", 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1439999999999997e-06 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292228+0000", 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59841 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.257893+0000", 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034743535999999998, 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037171566000000003, 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59841 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257893+0000", 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257893+0000", 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257892+0000", 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257895+0000", 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.931e-06 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257902+0000", 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6850000000000004e-06 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257906+0000", 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4270000000000001e-06 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.504 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59937 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260766+0000", 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031870598, 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034331426999999998, 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59937 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.505 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260766+0000", 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260766+0000", 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260765+0000", 2024-04-24T18:08:12.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260768+0000", 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4269999999999998e-06 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260775+0000", 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.99e-06 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260779+0000", 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9480000000000001e-06 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60066 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270201+0000", 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022435236000000001, 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.024935128000000001, 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60066 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270201+0000", 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270201+0000", 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270200+0000", 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270203+0000", 2024-04-24T18:08:12.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7020000000000001e-06 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270210+0000", 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4230000000000002e-06 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270214+0000", 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.002e-06 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60097 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276233+0000", 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016403081, 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018945048999999999, 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60097 2024-04-24T18:08:12.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276233+0000", 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276233+0000", 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276233+0000", 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276235+0000", 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5500000000000001e-06 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276241+0000", 2024-04-24T18:08:12.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0650000000000004e-06 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276245+0000", 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60195 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282309+0000", 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010327469000000001, 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012915611, 2024-04-24T18:08:12.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60195 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282309+0000", 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282309+0000", 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282308+0000", 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282311+0000", 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3250000000000002e-06 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282317+0000", 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0800000000000002e-06 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282321+0000", 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9879999999999998e-06 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60323 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287810+0000", 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0048258179999999999, 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0074526080000000003, 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.519 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60323 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287810+0000", 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287810+0000", 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287810+0000", 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287812+0000", 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.322e-06 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287818+0000", 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0510000000000002e-06 2024-04-24T18:08:12.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287822+0000", 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.687e-06 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60387 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292243+0000", 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00039350000000000002, 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0030574970000000002, 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60387 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292243+0000", 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292243+0000", 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292242+0000", 2024-04-24T18:08:12.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292248+0000", 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6919999999999999e-06 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292252+0000", 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3950000000000003e-06 2024-04-24T18:08:12.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292255+0000", 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3599999999999999e-06 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59842 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.257911+0000", 2024-04-24T18:08:12.526 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034725069999999997, 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037427199000000001, 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59842 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257911+0000", 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257911+0000", 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257911+0000", 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257914+0000", 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8289999999999999e-06 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257920+0000", 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4389999999999997e-06 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257924+0000", 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1790000000000001e-06 2024-04-24T18:08:12.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59938 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260783+0000", 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.031852964999999997, 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034593669000000001, 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59938 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260783+0000", 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260783+0000", 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260783+0000", 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260785+0000", 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.593e-06 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260792+0000", 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3609999999999997e-06 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260797+0000", 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3299999999999998e-06 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60067 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270218+0000", 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022418279999999999, 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.025212181, 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.533 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60067 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270218+0000", 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270218+0000", 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270217+0000", 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270220+0000", 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4779999999999998e-06 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270226+0000", 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.879e-06 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270230+0000", 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9230000000000004e-06 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60098 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276249+0000", 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016386696999999999, 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019219488999999999, 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60098 2024-04-24T18:08:12.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276249+0000", 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276249+0000", 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276249+0000", 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276251+0000", 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3039999999999999e-06 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276257+0000", 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0240000000000003e-06 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276261+0000", 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.817e-06 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60196 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282327+0000", 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010309647, 2024-04-24T18:08:12.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013176102, 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60196 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282327+0000", 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282327+0000", 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282326+0000", 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282329+0000", 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.464e-06 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282335+0000", 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1909999999999997e-06 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282340+0000", 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1250000000000001e-06 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60324 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.543 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287826+0000", 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0048098250000000002, 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.007710588, 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60324 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287826+0000", 2024-04-24T18:08:12.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287826+0000", 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287826+0000", 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287828+0000", 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.266e-06 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287834+0000", 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.947e-06 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287838+0000", 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4340000000000001e-06 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60388 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292257+0000", 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00037881900000000002, 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00331767, 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60388 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292257+0000", 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292257+0000", 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292257+0000", 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292259+0000", 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5799999999999999e-06 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292262+0000", 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9189999999999999e-06 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292264+0000", 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.086e-06 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59843 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.550 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.257929+0000", 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034707531999999999, 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037688034000000002, 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59843 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257929+0000", 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257929+0000", 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257928+0000", 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257931+0000", 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5280000000000002e-06 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257937+0000", 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.015e-06 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257941+0000", 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.766e-06 2024-04-24T18:08:12.553 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59939 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.262163+0000", 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.030473364999999999, 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.033492920000000002, 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.554 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59939 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262163+0000", 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262163+0000", 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260801+0000", 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9986386 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262167+0000", 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0013662349999999999 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262180+0000", 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2267e-05 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262186+0000", 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7499999999999997e-06 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60068 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.557 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270234+0000", 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022402179000000001, 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.025456304999999999, 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60068 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270234+0000", 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270234+0000", 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270234+0000", 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270236+0000", 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3250000000000002e-06 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270241+0000", 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5439999999999998e-06 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270247+0000", 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.631e-06 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60099 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276267+0000", 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016368975000000001, 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019484982000000001, 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60099 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276267+0000", 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276267+0000", 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276267+0000", 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276269+0000", 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6129999999999998e-06 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276275+0000", 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276279+0000", 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0459999999999999e-06 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60197 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282382+0000", 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010253923, 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013408966, 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60197 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282382+0000", 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282382+0000", 2024-04-24T18:08:12.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282382+0000", 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282386+0000", 2024-04-24T18:08:12.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9310000000000001e-06 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282394+0000", 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282399+0000", 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2329999999999997e-06 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60325 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287842+0000", 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0047940889999999996, 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0079881210000000008, 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60325 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287842+0000", 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287842+0000", 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287842+0000", 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287844+0000", 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3199999999999998e-06 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287851+0000", 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7469999999999999e-06 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287855+0000", 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0409999999999999e-06 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60389 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292266+0000", 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00037044600000000002, 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0036072370000000001, 2024-04-24T18:08:12.571 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60389 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292266+0000", 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292266+0000", 2024-04-24T18:08:12.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292265+0000", 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292267+0000", 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292270+0000", 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9409999999999999e-06 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292272+0000", 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0360000000000001e-06 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59844 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.257945+0000", 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034691246000000002, 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037973598999999997, 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59844 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257945+0000", 2024-04-24T18:08:12.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257945+0000", 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257945+0000", 2024-04-24T18:08:12.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257947+0000", 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8789999999999999e-06 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257954+0000", 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1199999999999999e-06 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257957+0000", 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8940000000000003e-06 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59940 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.262194+0000", 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.030441968, 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.033763345, 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59940 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262194+0000", 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262194+0000", 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262194+0000", 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262197+0000", 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1750000000000001e-06 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262204+0000", 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9319999999999999e-06 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262208+0000", 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6929999999999998e-06 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60069 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.581 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270295+0000", 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022341021999999999, 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.025702384000000002, 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60069 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270295+0000", 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270295+0000", 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270295+0000", 2024-04-24T18:08:12.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270299+0000", 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9729999999999999e-06 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270307+0000", 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3270000000000001e-06 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270311+0000", 2024-04-24T18:08:12.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5569999999999997e-06 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60100 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276284+0000", 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016352623, 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019757609999999998, 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.585 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60100 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276284+0000", 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276284+0000", 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276283+0000", 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276286+0000", 2024-04-24T18:08:12.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4820000000000001e-06 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276292+0000", 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0580000000000003e-06 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276296+0000", 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0899999999999998e-06 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60198 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282404+0000", 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010231825, 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013679073999999999, 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.589 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60198 2024-04-24T18:08:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282404+0000", 2024-04-24T18:08:12.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282404+0000", 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282404+0000", 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282407+0000", 2024-04-24T18:08:12.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.587e-06 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282413+0000", 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282417+0000", 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6619999999999996e-06 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.592 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60326 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287860+0000", 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.004775657, 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0082614249999999993, 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60326 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287860+0000", 2024-04-24T18:08:12.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287860+0000", 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287860+0000", 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287866+0000", 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8499999999999999e-06 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287872+0000", 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8619999999999999e-06 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287876+0000", 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0099999999999997e-06 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60390 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292274+0000", 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00036193299999999999, 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0038975110000000002, 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60390 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292274+0000", 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292274+0000", 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292274+0000", 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292275+0000", 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3909999999999999e-06 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292278+0000", 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8839999999999998e-06 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292280+0000", 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1100000000000001e-06 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59845 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.257962+0000", 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034673957999999998, 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038244552000000001, 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59845 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257962+0000", 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257962+0000", 2024-04-24T18:08:12.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257962+0000", 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257965+0000", 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.83e-06 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257971+0000", 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257974+0000", 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.659e-06 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59941 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.262213+0000", 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.030422793, 2024-04-24T18:08:12.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034033657000000002, 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59941 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262213+0000", 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262213+0000", 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262213+0000", 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262215+0000", 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4669999999999998e-06 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262222+0000", 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5259999999999999e-06 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262226+0000", 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2259999999999999e-06 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.605 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60070 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270317+0000", 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022319499999999999, 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.025965742999999999, 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60070 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.606 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270317+0000", 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270317+0000", 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270316+0000", 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270322+0000", 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6300000000000003e-06 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270328+0000", 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3679999999999998e-06 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270332+0000", 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8809999999999998e-06 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60101 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276302+0000", 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016334402000000001, 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020014865, 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.609 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60101 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276302+0000", 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276302+0000", 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276301+0000", 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276304+0000", 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7e-06 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276311+0000", 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.25e-06 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276315+0000", 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8299999999999998e-06 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60199 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.612 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282422+0000", 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010214373000000001, 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013929166, 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60199 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282422+0000", 2024-04-24T18:08:12.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282422+0000", 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282421+0000", 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282424+0000", 2024-04-24T18:08:12.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8310000000000001e-06 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282431+0000", 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3720000000000001e-06 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282435+0000", 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1239999999999998e-06 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.615 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60327 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287880+0000", 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0047559150000000003, 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0085090310000000002, 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60327 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287880+0000", 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287880+0000", 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287880+0000", 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287883+0000", 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.638e-06 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287889+0000", 2024-04-24T18:08:12.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9909999999999996e-06 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287894+0000", 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0600000000000001e-06 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60391 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292283+0000", 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00035350300000000002, 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0041398870000000001, 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60391 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292283+0000", 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292283+0000", 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292282+0000", 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292284+0000", 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8419999999999999e-06 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292287+0000", 2024-04-24T18:08:12.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0130000000000002e-06 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292289+0000", 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.623 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59846 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258002+0000", 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034634402000000002, 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038464326, 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59846 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258002+0000", 2024-04-24T18:08:12.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258002+0000", 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258001+0000", 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258005+0000", 2024-04-24T18:08:12.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3900000000000002e-06 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258012+0000", 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8150000000000006e-06 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258018+0000", 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4559999999999999e-06 2024-04-24T18:08:12.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59942 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.262231+0000", 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.030405488000000001, 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034277301000000003, 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59942 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262231+0000", 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262231+0000", 2024-04-24T18:08:12.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262230+0000", 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262233+0000", 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:12.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262239+0000", 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2859999999999996e-06 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262243+0000", 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3019999999999996e-06 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.630 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60071 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270337+0000", 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022299583000000001, 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026205257999999999, 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60071 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270337+0000", 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270337+0000", 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270336+0000", 2024-04-24T18:08:12.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270339+0000", 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.356e-06 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270344+0000", 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8030000000000002e-06 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270348+0000", 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7139999999999999e-06 2024-04-24T18:08:12.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60102 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276320+0000", 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016316614, 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020258109999999999, 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60102 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276320+0000", 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276320+0000", 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276319+0000", 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276321+0000", 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276329+0000", 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2810000000000003e-06 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276333+0000", 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9389999999999999e-06 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60200 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282439+0000", 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010196785999999999, 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014178837, 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.638 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60200 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282439+0000", 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282439+0000", 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282439+0000", 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282442+0000", 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4889999999999998e-06 2024-04-24T18:08:12.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282449+0000", 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9200000000000004e-06 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282454+0000", 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1500000000000001e-06 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.641 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60328 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287898+0000", 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0047383039999999996, 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0087531650000000003, 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60328 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.642 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287898+0000", 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287898+0000", 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287897+0000", 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287900+0000", 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3250000000000002e-06 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287906+0000", 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8039999999999998e-06 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287909+0000", 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60392 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292291+0000", 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.000345046, 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0043934029999999997, 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.645 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60392 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292291+0000", 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292291+0000", 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292291+0000", 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292292+0000", 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3570000000000001e-06 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292295+0000", 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8320000000000002e-06 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292297+0000", 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0719999999999998e-06 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59847 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258023+0000", 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034613377000000001, 2024-04-24T18:08:12.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038700052999999998, 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59847 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258023+0000", 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258023+0000", 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258022+0000", 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258025+0000", 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0750000000000002e-06 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258032+0000", 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.64e-06 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258036+0000", 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3189999999999996e-06 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59943 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.262249+0000", 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.030386765999999999, 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034507050999999997, 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59943 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262249+0000", 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262249+0000", 2024-04-24T18:08:12.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262249+0000", 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262252+0000", 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262258+0000", 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2920000000000001e-06 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262262+0000", 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60072 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270352+0000", 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022283879999999999, 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026438438000000002, 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60072 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270352+0000", 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270352+0000", 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270352+0000", 2024-04-24T18:08:12.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270354+0000", 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2239999999999998e-06 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270361+0000", 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5729999999999996e-06 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270364+0000", 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7060000000000002e-06 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60103 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276337+0000", 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016299218000000001, 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020489714999999999, 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60103 2024-04-24T18:08:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276337+0000", 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276337+0000", 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276337+0000", 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276339+0000", 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.357e-06 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276345+0000", 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.891e-06 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276349+0000", 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.743e-06 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.663 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60201 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282458+0000", 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010177891, 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014401565, 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60201 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282458+0000", 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282458+0000", 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282458+0000", 2024-04-24T18:08:12.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282464+0000", 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7889999999999997e-06 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282471+0000", 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2960000000000004e-06 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282475+0000", 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2429999999999999e-06 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60329 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287913+0000", 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0047227349999999996, 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089802200000000006, 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.667 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60329 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287913+0000", 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287913+0000", 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287913+0000", 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287915+0000", 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1260000000000001e-06 2024-04-24T18:08:12.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287921+0000", 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287924+0000", 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.343e-06 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60393 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292299+0000", 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00033696300000000002, 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0046268250000000002, 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60393 2024-04-24T18:08:12.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292299+0000", 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292299+0000", 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292299+0000", 2024-04-24T18:08:12.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292300+0000", 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3829999999999999e-06 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292303+0000", 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8499999999999998e-06 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292305+0000", 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9879999999999999e-06 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59848 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258041+0000", 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034595229999999998, 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038920036999999998, 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.674 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59848 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258041+0000", 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258041+0000", 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258040+0000", 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258043+0000", 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258049+0000", 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1659999999999999e-06 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258054+0000", 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2699999999999998e-06 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59944 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.262266+0000", 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.030369858, 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034729387, 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59944 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262266+0000", 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262266+0000", 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262266+0000", 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262273+0000", 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.691e-06 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262279+0000", 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5860000000000001e-06 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262284+0000", 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3290000000000004e-06 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60073 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270369+0000", 2024-04-24T18:08:12.680 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022267413999999999, 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026660526, 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60073 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270369+0000", 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270369+0000", 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270368+0000", 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270371+0000", 2024-04-24T18:08:12.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3429999999999998e-06 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270376+0000", 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270381+0000", 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4150000000000001e-06 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.683 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60104 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276353+0000", 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016283414, 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020716795, 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60104 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276353+0000", 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276353+0000", 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276352+0000", 2024-04-24T18:08:12.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276355+0000", 2024-04-24T18:08:12.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1579999999999999e-06 2024-04-24T18:08:12.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276360+0000", 2024-04-24T18:08:12.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:08:12.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276364+0000", 2024-04-24T18:08:12.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4829999999999999e-06 2024-04-24T18:08:12.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.689 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.689 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60202 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.689 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282490+0000", 2024-04-24T18:08:12.689 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010146031, 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014612899, 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60202 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.690 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282490+0000", 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282490+0000", 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282490+0000", 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282493+0000", 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0299999999999998e-06 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282500+0000", 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8940000000000004e-06 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282504+0000", 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2429999999999999e-06 2024-04-24T18:08:12.692 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60330 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287928+0000", 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.004707864, 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0092120359999999998, 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60330 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287928+0000", 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287928+0000", 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287928+0000", 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287930+0000", 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2350000000000002e-06 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287937+0000", 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9460000000000001e-06 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287941+0000", 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.693e-06 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60394 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292307+0000", 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00032888200000000001, 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0048911780000000004, 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60394 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292307+0000", 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292307+0000", 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292307+0000", 2024-04-24T18:08:12.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292308+0000", 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.322e-06 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292311+0000", 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7939999999999998e-06 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292313+0000", 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9240000000000001e-06 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59849 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258059+0000", 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.034577484999999998, 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039183570000000001, 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.700 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59849 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258059+0000", 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258059+0000", 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258058+0000", 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258061+0000", 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6419999999999999e-06 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258068+0000", 2024-04-24T18:08:12.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9410000000000001e-06 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258072+0000", 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0459999999999999e-06 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59945 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.262289+0000", 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.030347562000000002, 2024-04-24T18:08:12.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.034988106999999997, 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59945 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262289+0000", 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262289+0000", 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262288+0000", 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262291+0000", 2024-04-24T18:08:12.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6479999999999999e-06 2024-04-24T18:08:12.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262297+0000", 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.161e-06 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.262301+0000", 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0929999999999996e-06 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60074 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270385+0000", 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022250952000000001, 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026924219999999999, 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60074 2024-04-24T18:08:12.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270385+0000", 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270385+0000", 2024-04-24T18:08:12.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270385+0000", 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270387+0000", 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270393+0000", 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8839999999999999e-06 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270397+0000", 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7110000000000001e-06 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60105 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276368+0000", 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016267782000000001, 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020977584000000001, 2024-04-24T18:08:12.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60105 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276368+0000", 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276368+0000", 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276368+0000", 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276370+0000", 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0609999999999998e-06 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276376+0000", 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6829999999999996e-06 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276379+0000", 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.608e-06 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60203 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282508+0000", 2024-04-24T18:08:12.714 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010127730999999999, 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014870326999999999, 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60203 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282508+0000", 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282508+0000", 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282508+0000", 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282510+0000", 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282516+0000", 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.891e-06 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282520+0000", 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6990000000000001e-06 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60331 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287945+0000", 2024-04-24T18:08:12.717 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0046908949999999996, 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094655179999999992, 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60331 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287945+0000", 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287945+0000", 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287945+0000", 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287947+0000", 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4830000000000002e-06 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287953+0000", 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9270000000000002e-06 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287957+0000", 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0149999999999997e-06 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60395 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292315+0000", 2024-04-24T18:08:12.720 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.000320966, 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0051281399999999998, 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60395 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292315+0000", 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292315+0000", 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292315+0000", 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292316+0000", 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.296e-06 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292319+0000", 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8729999999999999e-06 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292321+0000", 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.012e-06 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59914 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.723 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258697+0000", 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033939592999999997, 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038781508999999999, 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59914 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258697+0000", 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258697+0000", 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258696+0000", 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258699+0000", 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258705+0000", 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1419999999999998e-06 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258709+0000", 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6310000000000002e-06 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60043 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.726 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.267747+0000", 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024888947000000002, 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.029769802000000001, 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60043 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267747+0000", 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267747+0000", 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267747+0000", 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267749+0000", 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.509e-06 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267756+0000", 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3269999999999997e-06 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267760+0000", 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60075 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.729 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270402+0000", 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022233876999999999, 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027149296, 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60075 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270402+0000", 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270402+0000", 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270402+0000", 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270404+0000", 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1859999999999999e-06 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270410+0000", 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9989999999999997e-06 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270414+0000", 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8879999999999999e-06 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60106 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.732 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.276423+0000", 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.016213301999999999, 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.021162528, 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60106 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276423+0000", 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276423+0000", 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276422+0000", 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276428+0000", 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.259e-06 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276434+0000", 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3199999999999996e-06 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.276437+0000", 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0259999999999999e-06 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60204 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.735 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.282524+0000", 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.010112077000000001, 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.015093775, 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60204 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282524+0000", 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282524+0000", 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282524+0000", 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282526+0000", 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0949999999999998e-06 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282532+0000", 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6930000000000004e-06 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.282535+0000", 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5640000000000001e-06 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60332 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.738 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287962+0000", 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0046744250000000003, 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0097068710000000006, 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60332 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287962+0000", 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287962+0000", 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287961+0000", 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287964+0000", 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:12.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287969+0000", 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287973+0000", 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4079999999999998e-06 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60396 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.741 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292323+0000", 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.000312916, 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0053805260000000001, 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60396 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292323+0000", 2024-04-24T18:08:12.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292323+0000", 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292323+0000", 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292324+0000", 2024-04-24T18:08:12.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3379999999999999e-06 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292328+0000", 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5420000000000001e-06 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292330+0000", 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2060000000000001e-06 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.744 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59915 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258715+0000", 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033921536000000002, 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039023585999999999, 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59915 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258715+0000", 2024-04-24T18:08:12.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258715+0000", 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258714+0000", 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258717+0000", 2024-04-24T18:08:12.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4030000000000001e-06 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258723+0000", 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9900000000000002e-06 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258726+0000", 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6780000000000002e-06 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60044 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.267764+0000", 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024872123999999999, 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.030009448000000001, 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60044 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267764+0000", 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267764+0000", 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267764+0000", 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267766+0000", 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3609999999999999e-06 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267771+0000", 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4569999999999996e-06 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267775+0000", 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9750000000000001e-06 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60076 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.270418+0000", 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.022217822000000002, 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027391334999999999, 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60076 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270418+0000", 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270418+0000", 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270418+0000", 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270420+0000", 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.967e-06 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270426+0000", 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9909999999999999e-06 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270430+0000", 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5439999999999999e-06 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.753 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60173 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280571+0000", 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.012065026, 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.017273059, 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60173 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.754 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280571+0000", 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280571+0000", 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280571+0000", 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280573+0000", 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.39e-06 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280579+0000", 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4289999999999998e-06 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280583+0000", 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7330000000000001e-06 2024-04-24T18:08:12.756 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60301 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.284524+0000", 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0081121990000000005, 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013355989, 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60301 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.757 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284524+0000", 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284524+0000", 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284524+0000", 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284526+0000", 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1569999999999998e-06 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284532+0000", 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7640000000000002e-06 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284535+0000", 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.692e-06 2024-04-24T18:08:12.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60365 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290438+0000", 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0021984650000000001, 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0074756739999999999, 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60365 2024-04-24T18:08:12.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290438+0000", 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290438+0000", 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290437+0000", 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290440+0000", 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.385e-06 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290445+0000", 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5280000000000003e-06 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290450+0000", 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5569999999999997e-06 2024-04-24T18:08:12.762 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60429 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292332+0000", 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00030384299999999998, 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00561358, 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60429 2024-04-24T18:08:12.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292332+0000", 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292332+0000", 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292332+0000", 2024-04-24T18:08:12.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292334+0000", 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3990000000000001e-06 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292336+0000", 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.881e-06 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292338+0000", 2024-04-24T18:08:12.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9180000000000001e-06 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59916 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258731+0000", 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033905414000000002, 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039249599000000003, 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.766 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59916 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258731+0000", 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258731+0000", 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258730+0000", 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258733+0000", 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.402e-06 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258739+0000", 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7950000000000004e-06 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258742+0000", 2024-04-24T18:08:12.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.512e-06 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60045 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.267780+0000", 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024856244, 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.030237140999999999, 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.769 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60045 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267780+0000", 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267780+0000", 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267780+0000", 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267782+0000", 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2979999999999999e-06 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267790+0000", 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0010000000000001e-06 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267794+0000", 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8619999999999996e-06 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60174 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280588+0000", 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.012047950999999999, 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.017466045999999999, 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.772 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60174 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280588+0000", 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280588+0000", 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280588+0000", 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280590+0000", 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.187e-06 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280596+0000", 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7599999999999999e-06 2024-04-24T18:08:12.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280599+0000", 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.614e-06 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60302 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.284543+0000", 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0080927689999999997, 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013545934000000001, 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60302 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284543+0000", 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284543+0000", 2024-04-24T18:08:12.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284543+0000", 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284545+0000", 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5220000000000002e-06 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284551+0000", 2024-04-24T18:08:12.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8680000000000004e-06 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284555+0000", 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5779999999999999e-06 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60366 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290454+0000", 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002181788, 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0076708890000000002, 2024-04-24T18:08:12.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60366 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290454+0000", 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290454+0000", 2024-04-24T18:08:12.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290454+0000", 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290456+0000", 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3499999999999999e-06 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290464+0000", 2024-04-24T18:08:12.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6899999999999992e-06 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290468+0000", 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9040000000000002e-06 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60430 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292340+0000", 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00029584799999999998, 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0058257969999999997, 2024-04-24T18:08:12.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60430 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292340+0000", 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292340+0000", 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292340+0000", 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292341+0000", 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3e-06 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292344+0000", 2024-04-24T18:08:12.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.852e-06 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292346+0000", 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9039999999999999e-06 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59917 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258769+0000", 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033866712, 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039432983999999997, 2024-04-24T18:08:12.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59917 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258769+0000", 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258769+0000", 2024-04-24T18:08:12.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258769+0000", 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258772+0000", 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2569999999999999e-06 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258780+0000", 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5909999999999998e-06 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258784+0000", 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.549e-06 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60046 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.267802+0000", 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024833847999999999, 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.030442008, 2024-04-24T18:08:12.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60046 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267802+0000", 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267802+0000", 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267802+0000", 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267805+0000", 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9639999999999999e-06 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267811+0000", 2024-04-24T18:08:12.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0959999999999997e-06 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267815+0000", 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60077 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272160+0000", 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020475883, 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026119184, 2024-04-24T18:08:12.790 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60077 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272160+0000", 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272160+0000", 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270433+0000", 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9982729 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272164+0000", 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.001730931 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272172+0000", 2024-04-24T18:08:12.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1829999999999994e-06 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272177+0000", 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.437e-06 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60175 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280604+0000", 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.012031783000000001, 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.01773543, 2024-04-24T18:08:12.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60175 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280604+0000", 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280604+0000", 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280604+0000", 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280606+0000", 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2929999999999999e-06 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280612+0000", 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7899999999999996e-06 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280616+0000", 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7189999999999999e-06 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60303 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.284559+0000", 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0080770700000000004, 2024-04-24T18:08:12.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013816479, 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60303 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284559+0000", 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284559+0000", 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284559+0000", 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284561+0000", 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.305e-06 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284567+0000", 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0279999999999997e-06 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284571+0000", 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6739999999999999e-06 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60367 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290472+0000", 2024-04-24T18:08:12.799 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0021638019999999998, 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0079385240000000006, 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60367 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290472+0000", 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290472+0000", 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290472+0000", 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290474+0000", 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.193e-06 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290480+0000", 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7610000000000004e-06 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290483+0000", 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.507e-06 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60431 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.802 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292348+0000", 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00028798700000000002, 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060984979999999999, 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60431 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292348+0000", 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292348+0000", 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292348+0000", 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292351+0000", 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5149999999999998e-06 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292355+0000", 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4429999999999999e-06 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292358+0000", 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5760000000000002e-06 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59918 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258789+0000", 2024-04-24T18:08:12.805 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033846838999999997, 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039694343999999999, 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59918 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258789+0000", 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258789+0000", 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258789+0000", 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258792+0000", 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258798+0000", 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1369999999999999e-06 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258802+0000", 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60047 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.808 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.267820+0000", 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024816052000000002, 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.030698534999999999, 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60047 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267820+0000", 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267820+0000", 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267820+0000", 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267822+0000", 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6910000000000002e-06 2024-04-24T18:08:12.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267829+0000", 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3620000000000002e-06 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267833+0000", 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6660000000000001e-06 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.811 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60078 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272181+0000", 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020454849000000001, 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026379447, 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60078 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272181+0000", 2024-04-24T18:08:12.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272181+0000", 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272181+0000", 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272183+0000", 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0719999999999998e-06 2024-04-24T18:08:12.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272187+0000", 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.428e-06 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272189+0000", 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3470000000000001e-06 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.814 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60176 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280620+0000", 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.012016377, 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.017976095000000001, 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60176 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280620+0000", 2024-04-24T18:08:12.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280620+0000", 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280619+0000", 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280622+0000", 2024-04-24T18:08:12.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.385e-06 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280627+0000", 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3700000000000003e-06 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280631+0000", 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7390000000000001e-06 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.817 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60304 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.284576+0000", 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0080598500000000003, 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014056279, 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60304 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284576+0000", 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284576+0000", 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284576+0000", 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284579+0000", 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6429999999999999e-06 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284586+0000", 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6790000000000005e-06 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284590+0000", 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1160000000000001e-06 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60368 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290488+0000", 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002148334, 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.008180049, 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60368 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290488+0000", 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290488+0000", 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290487+0000", 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290490+0000", 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8310000000000001e-06 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290496+0000", 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.28e-06 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290500+0000", 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.805e-06 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60432 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292378+0000", 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00025836499999999998, 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0063242760000000002, 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60432 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292378+0000", 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292378+0000", 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292377+0000", 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292380+0000", 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0690000000000002e-06 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292387+0000", 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.618e-06 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292391+0000", 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.189e-06 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59919 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258806+0000", 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033829959, 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039942967000000003, 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59919 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258806+0000", 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258806+0000", 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258806+0000", 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258808+0000", 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2699999999999999e-06 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258814+0000", 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8769999999999998e-06 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258818+0000", 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.907e-06 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60048 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.267837+0000", 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024799195999999999, 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.030947703, 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60048 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267837+0000", 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267837+0000", 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267837+0000", 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267839+0000", 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2510000000000001e-06 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267845+0000", 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1380000000000004e-06 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267849+0000", 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60079 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272218+0000", 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020418262, 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026601518000000001, 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60079 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272218+0000", 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272218+0000", 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272217+0000", 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272222+0000", 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272233+0000", 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1353e-05 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272238+0000", 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.271e-06 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.835 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60177 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280635+0000", 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.012000827, 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018219511000000001, 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60177 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280635+0000", 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280635+0000", 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280635+0000", 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280642+0000", 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6490000000000002e-06 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280648+0000", 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1510000000000001e-06 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280652+0000", 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7740000000000004e-06 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.838 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60305 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.284595+0000", 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0080410770000000006, 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.014293671000000001, 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60305 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284595+0000", 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284595+0000", 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284595+0000", 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284597+0000", 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.655e-06 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284604+0000", 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3720000000000001e-06 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284608+0000", 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9079999999999997e-06 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60369 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290505+0000", 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0021314469999999999, 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.008418132, 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60369 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290505+0000", 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290505+0000", 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290504+0000", 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290507+0000", 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.464e-06 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290513+0000", 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9399999999999999e-06 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290517+0000", 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6550000000000003e-06 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60433 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292396+0000", 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00024063399999999999, 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065731629999999999, 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.845 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60433 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292396+0000", 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292396+0000", 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292395+0000", 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292397+0000", 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2900000000000001e-06 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292403+0000", 2024-04-24T18:08:12.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9630000000000003e-06 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292407+0000", 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.907e-06 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59920 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.258824+0000", 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033812504, 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.040179279999999998, 2024-04-24T18:08:12.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59920 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258824+0000", 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258824+0000", 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258823+0000", 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258830+0000", 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6830000000000002e-06 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258837+0000", 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1670000000000002e-06 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258842+0000", 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4070000000000003e-06 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60049 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268358+0000", 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024277796000000001, 2024-04-24T18:08:12.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.030680307, 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60049 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268358+0000", 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268358+0000", 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268358+0000", 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268362+0000", 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1269999999999996e-06 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268370+0000", 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1869999999999997e-06 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268375+0000", 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4500000000000003e-06 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60080 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272245+0000", 2024-04-24T18:08:12.854 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020391256, 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0268293, 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60080 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272245+0000", 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272245+0000", 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272244+0000", 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272247+0000", 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8250000000000001e-06 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272253+0000", 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9909999999999999e-06 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272257+0000", 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7189999999999999e-06 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60178 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280657+0000", 2024-04-24T18:08:12.857 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011978931999999999, 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018449980000000001, 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60178 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280657+0000", 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280657+0000", 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280657+0000", 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280659+0000", 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7199999999999998e-06 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280666+0000", 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.049e-06 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280670+0000", 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8590000000000004e-06 2024-04-24T18:08:12.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60306 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287542+0000", 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0050941199999999997, 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011605837000000001, 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60306 2024-04-24T18:08:12.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287542+0000", 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287542+0000", 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.284612+0000", 2024-04-24T18:08:12.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9970694 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287547+0000", 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0029352179999999999 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287556+0000", 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9900000000000003e-06 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287562+0000", 2024-04-24T18:08:12.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6259999999999997e-06 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60370 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290522+0000", 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0021145339999999999, 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0086675420000000003, 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.864 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60370 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290522+0000", 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290522+0000", 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290521+0000", 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290524+0000", 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3309999999999998e-06 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290530+0000", 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0360000000000003e-06 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290533+0000", 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9330000000000003e-06 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60434 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292412+0000", 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.000224613, 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0068127769999999999, 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60434 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292412+0000", 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292412+0000", 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292411+0000", 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292413+0000", 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1100000000000001e-06 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292419+0000", 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0059999999999998e-06 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292423+0000", 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8089999999999999e-06 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59921 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.259095+0000", 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.033541242999999998, 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.040169822000000001, 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59921 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.259095+0000", 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.259095+0000", 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.258846+0000", 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9997511 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.259099+0000", 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00025305599999999998 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.259110+0000", 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0709e-05 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.259116+0000", 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9889999999999997e-06 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60050 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268381+0000", 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024255549000000001, 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.030918602, 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60050 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268381+0000", 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268381+0000", 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268380+0000", 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268383+0000", 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6620000000000001e-06 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268389+0000", 2024-04-24T18:08:12.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5509999999999996e-06 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268395+0000", 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7599999999999999e-06 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60081 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272262+0000", 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020374384999999998, 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027071333999999999, 2024-04-24T18:08:12.876 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60081 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272262+0000", 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272262+0000", 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272261+0000", 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272264+0000", 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5809999999999999e-06 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272270+0000", 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.879e-06 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272274+0000", 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9129999999999996e-06 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60179 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280675+0000", 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011960894999999999, 2024-04-24T18:08:12.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018692446000000001, 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60179 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280675+0000", 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280675+0000", 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280675+0000", 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280677+0000", 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5059999999999998e-06 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280684+0000", 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1299999999999998e-06 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280687+0000", 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9439999999999998e-06 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60307 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287570+0000", 2024-04-24T18:08:12.882 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0050660760000000001, 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.011831105, 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60307 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287570+0000", 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287570+0000", 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287569+0000", 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287572+0000", 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7839999999999999e-06 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287579+0000", 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4609999999999996e-06 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287583+0000", 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60371 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290538+0000", 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0020986099999999999, 2024-04-24T18:08:12.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0089028779999999995, 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60371 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290538+0000", 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290538+0000", 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290537+0000", 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290539+0000", 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.119e-06 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290546+0000", 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2940000000000002e-06 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290549+0000", 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.799e-06 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60435 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.888 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.292428+0000", 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00020863899999999999, 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0070477860000000003, 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60435 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292428+0000", 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292428+0000", 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292427+0000", 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292429+0000", 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:08:12.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292435+0000", 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6049999999999997e-06 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292439+0000", 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7440000000000001e-06 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.891 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59922 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260162+0000", 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.032473835999999999, 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039351601999999999, 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59922 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260162+0000", 2024-04-24T18:08:12.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260162+0000", 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.259122+0000", 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9989595 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260167+0000", 2024-04-24T18:08:12.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0010452720000000001 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260181+0000", 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3497e-05 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260188+0000", 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.379e-06 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60051 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268400+0000", 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024236321000000002, 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.031150509, 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60051 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268400+0000", 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268400+0000", 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268399+0000", 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268402+0000", 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5890000000000001e-06 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268408+0000", 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1820000000000003e-06 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268414+0000", 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8969999999999996e-06 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60082 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272278+0000", 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020358304000000001, 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027306591000000002, 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60082 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272278+0000", 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272278+0000", 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272278+0000", 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272284+0000", 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272291+0000", 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9090000000000003e-06 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272295+0000", 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.900 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60180 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280692+0000", 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011944375, 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.018930957000000002, 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60180 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.901 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280692+0000", 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280692+0000", 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280691+0000", 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280694+0000", 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4439999999999998e-06 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280700+0000", 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7599999999999999e-06 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280703+0000", 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.844e-06 2024-04-24T18:08:12.903 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60308 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287587+0000", 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0050491959999999997, 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012070060000000001, 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60308 2024-04-24T18:08:12.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287587+0000", 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287587+0000", 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287587+0000", 2024-04-24T18:08:12.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287589+0000", 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3199999999999998e-06 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287595+0000", 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287598+0000", 2024-04-24T18:08:12.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4089999999999999e-06 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60372 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290565+0000", 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0020713060000000002, 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0091252290000000003, 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60372 2024-04-24T18:08:12.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290565+0000", 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290565+0000", 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290564+0000", 2024-04-24T18:08:12.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290567+0000", 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290574+0000", 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7909999999999999e-06 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290579+0000", 2024-04-24T18:08:12.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9749999999999998e-06 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60436 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294161+0000", 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9984751, 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0055648649999999996, 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60436 2024-04-24T18:08:12.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294161+0000", 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294161+0000", 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.292443+0000", 2024-04-24T18:08:12.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9982815 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294165+0000", 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0017228600000000001 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294176+0000", 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0818e-05 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294182+0000", 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0220000000000001e-06 2024-04-24T18:08:12.912 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59923 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 300032~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.260195+0000", 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.032440745, 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039579252000000002, 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.913 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59923 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260195+0000", 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260195+0000", 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260195+0000", 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260198+0000", 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2219999999999998e-06 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260206+0000", 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.157e-06 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.260211+0000", 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.8840000000000002e-06 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.299725+0000", 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039513980999999997 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.299764+0000", 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9022e-05 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60052 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268419+0000", 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024217343999999998, 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.031409871999999998, 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60052 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268419+0000", 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268419+0000", 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268418+0000", 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268421+0000", 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5579999999999999e-06 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268427+0000", 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3419999999999996e-06 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268431+0000", 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60083 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272300+0000", 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020336295000000001, 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027567464, 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60083 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272300+0000", 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272300+0000", 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272299+0000", 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272302+0000", 2024-04-24T18:08:12.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7140000000000002e-06 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272308+0000", 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0329999999999997e-06 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272312+0000", 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60181 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280719+0000", 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011917527000000001, 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019183370000000002, 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60181 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280719+0000", 2024-04-24T18:08:12.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280719+0000", 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280718+0000", 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280721+0000", 2024-04-24T18:08:12.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9229999999999998e-06 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280728+0000", 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9110000000000004e-06 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280732+0000", 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3749999999999996e-06 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.925 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60309 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287602+0000", 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.005034185, 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012343196000000001, 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60309 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287602+0000", 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287602+0000", 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287602+0000", 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287604+0000", 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.075e-06 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287609+0000", 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0540000000000002e-06 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287612+0000", 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60373 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290583+0000", 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002052824, 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0094002909999999999, 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60373 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290583+0000", 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290583+0000", 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290583+0000", 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290585+0000", 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290591+0000", 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1240000000000002e-06 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290596+0000", 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0600000000000001e-06 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.931 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60437 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294191+0000", 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9984446, 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0058260609999999996, 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60437 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294191+0000", 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294191+0000", 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294191+0000", 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294194+0000", 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.948e-06 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294201+0000", 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1670000000000002e-06 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294206+0000", 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6709999999999998e-06 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.934 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60053 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268436+0000", 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024200337999999998, 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.031622201000000003, 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60053 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.935 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268436+0000", 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268436+0000", 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268435+0000", 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268438+0000", 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268444+0000", 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2210000000000002e-06 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268449+0000", 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5809999999999998e-06 2024-04-24T18:08:12.937 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60084 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272317+0000", 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020319642999999998, 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027776083999999999, 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.938 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60084 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272317+0000", 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272317+0000", 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272316+0000", 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272319+0000", 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.503e-06 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272324+0000", 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.468e-06 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272328+0000", 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7120000000000002e-06 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60182 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280737+0000", 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011899465999999999, 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019391226000000001, 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.941 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60182 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280737+0000", 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280737+0000", 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280736+0000", 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280739+0000", 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280744+0000", 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6470000000000003e-06 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280750+0000", 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2669999999999997e-06 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60310 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287616+0000", 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0050199600000000004, 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.01255292, 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.944 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60310 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287616+0000", 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287616+0000", 2024-04-24T18:08:12.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287616+0000", 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287618+0000", 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3120000000000001e-06 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287624+0000", 2024-04-24T18:08:12.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2820000000000004e-06 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287627+0000", 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6169999999999998e-06 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60374 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290600+0000", 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0020365819999999999, 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0096057820000000002, 2024-04-24T18:08:12.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60374 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290600+0000", 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290600+0000", 2024-04-24T18:08:12.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290599+0000", 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290605+0000", 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1469999999999998e-06 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290612+0000", 2024-04-24T18:08:12.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290616+0000", 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60438 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294226+0000", 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9984107, 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0060145490000000001, 2024-04-24T18:08:12.950 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60438 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294226+0000", 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294226+0000", 2024-04-24T18:08:12.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294225+0000", 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294228+0000", 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5769999999999998e-06 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294236+0000", 2024-04-24T18:08:12.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7670000000000003e-06 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294240+0000", 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8689999999999997e-06 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60054 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268453+0000", 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024183054999999998, 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.031822827999999997, 2024-04-24T18:08:12.953 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60054 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268453+0000", 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268453+0000", 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268453+0000", 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268455+0000", 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268461+0000", 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0569999999999998e-06 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268465+0000", 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9759999999999997e-06 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60085 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.272333+0000", 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.020303496000000001, 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027991412, 2024-04-24T18:08:12.956 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60085 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272333+0000", 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272333+0000", 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272332+0000", 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272334+0000", 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.204e-06 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272338+0000", 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4599999999999999e-06 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272341+0000", 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3380000000000001e-06 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60183 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280754+0000", 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011882063999999999, 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019604379000000002, 2024-04-24T18:08:12.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60183 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280754+0000", 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280754+0000", 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280754+0000", 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280756+0000", 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5330000000000001e-06 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280763+0000", 2024-04-24T18:08:12.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3260000000000001e-06 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280767+0000", 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9330000000000003e-06 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60311 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287631+0000", 2024-04-24T18:08:12.962 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0050049580000000003, 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.012761829000000001, 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60311 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287631+0000", 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287631+0000", 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287631+0000", 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287633+0000", 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4080000000000001e-06 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287640+0000", 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3459999999999999e-06 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287643+0000", 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6440000000000002e-06 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60375 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290620+0000", 2024-04-24T18:08:12.965 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.002015855, 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0098074240000000003, 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60375 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290620+0000", 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290620+0000", 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290620+0000", 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290622+0000", 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.587e-06 2024-04-24T18:08:12.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290628+0000", 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9429999999999997e-06 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290632+0000", 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5659999999999998e-06 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.968 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60439 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294244+0000", 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9983916, 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0062221050000000003, 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60439 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294244+0000", 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294244+0000", 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294244+0000", 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294247+0000", 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.841e-06 2024-04-24T18:08:12.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294254+0000", 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6079999999999998e-06 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294258+0000", 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9620000000000004e-06 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.971 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60055 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268470+0000", 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024166588999999999, 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.032032097000000002, 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60055 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268470+0000", 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268470+0000", 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268469+0000", 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268472+0000", 2024-04-24T18:08:12.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6079999999999998e-06 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268478+0000", 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9710000000000001e-06 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268482+0000", 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8480000000000003e-06 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60086 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274160+0000", 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018476052999999999, 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026405964000000001, 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60086 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274160+0000", 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274160+0000", 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.272345+0000", 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9981847 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274163+0000", 2024-04-24T18:08:12.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0018186249999999999 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274171+0000", 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8390000000000007e-06 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274175+0000", 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7919999999999999e-06 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60184 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280771+0000", 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011865044999999999, 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.019867289, 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60184 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280771+0000", 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280771+0000", 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280771+0000", 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280773+0000", 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.469e-06 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280779+0000", 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1449999999999996e-06 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280783+0000", 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60312 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287647+0000", 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0049886669999999996, 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013061164, 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60312 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287647+0000", 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287647+0000", 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287647+0000", 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287650+0000", 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4049999999999998e-06 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287656+0000", 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1650000000000003e-06 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287659+0000", 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.168e-06 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60376 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290636+0000", 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0020000420000000001, 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010118276000000001, 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60376 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290636+0000", 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290636+0000", 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290636+0000", 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290638+0000", 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3700000000000002e-06 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290644+0000", 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6799999999999998e-06 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290648+0000", 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8139999999999998e-06 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60440 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294263+0000", 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.998373, 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0065313979999999999, 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60440 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294263+0000", 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294263+0000", 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294263+0000", 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294265+0000", 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8670000000000001e-06 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294268+0000", 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5829999999999998e-06 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294270+0000", 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1610000000000001e-06 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60056 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268487+0000", 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024148724999999999, 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.032350413000000001, 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60056 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268487+0000", 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268487+0000", 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268487+0000", 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268490+0000", 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5110000000000002e-06 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268495+0000", 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9120000000000003e-06 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268499+0000", 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7919999999999999e-06 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60087 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274179+0000", 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018456984999999999, 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026708533999999999, 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60087 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274179+0000", 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274179+0000", 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274179+0000", 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274181+0000", 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.637e-06 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274184+0000", 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1889999999999999e-06 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274186+0000", 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60185 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280788+0000", 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011847896, 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020134272000000002, 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60185 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280788+0000", 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280788+0000", 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280788+0000", 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:12.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280790+0000", 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3700000000000002e-06 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280796+0000", 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9959999999999999e-06 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280801+0000", 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5179999999999998e-06 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:12.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60313 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287681+0000", 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0049552830000000004, 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013274927000000001, 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60313 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:12.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287681+0000", 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287681+0000", 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287680+0000", 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287683+0000", 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0510000000000001e-06 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287690+0000", 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2009999999999996e-06 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287694+0000", 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1930000000000003e-06 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60377 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290652+0000", 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0019840399999999998, 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010337769, 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60377 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290652+0000", 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290652+0000", 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290652+0000", 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290654+0000", 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3659999999999999e-06 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290660+0000", 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.857e-06 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290665+0000", 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3410000000000002e-06 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60441 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294273+0000", 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9983635, 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0067570340000000003, 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60441 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294273+0000", 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294273+0000", 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294273+0000", 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294274+0000", 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.325e-06 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294277+0000", 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1590000000000002e-06 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294279+0000", 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.875e-06 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.007 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60057 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268503+0000", 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024132773999999999, 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.032563547999999998, 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60057 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268503+0000", 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268503+0000", 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268503+0000", 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268505+0000", 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4310000000000001e-06 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268511+0000", 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268515+0000", 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4939999999999999e-06 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.010 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60088 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274188+0000", 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018447984000000001, 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.026929216999999998, 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60088 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274188+0000", 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274188+0000", 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274188+0000", 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274189+0000", 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4580000000000001e-06 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274192+0000", 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.909e-06 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274195+0000", 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3970000000000001e-06 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.013 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60186 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280805+0000", 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011830812, 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020346902, 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60186 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280805+0000", 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280805+0000", 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280805+0000", 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280808+0000", 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6680000000000001e-06 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280814+0000", 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1890000000000004e-06 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280818+0000", 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8979999999999998e-06 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.016 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60314 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287699+0000", 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0049375249999999999, 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013493834999999999, 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60314 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287699+0000", 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287699+0000", 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287698+0000", 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287704+0000", 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.879e-06 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287708+0000", 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.693e-06 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287710+0000", 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3369999999999998e-06 2024-04-24T18:08:13.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60378 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290670+0000", 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0019663570000000002, 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010558517, 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60378 2024-04-24T18:08:13.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290670+0000", 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290670+0000", 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290669+0000", 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290672+0000", 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2079999999999999e-06 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290677+0000", 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6529999999999999e-06 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290681+0000", 2024-04-24T18:08:13.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5530000000000001e-06 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60442 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294281+0000", 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9983549, 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0069849100000000004, 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.023 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60442 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294281+0000", 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294281+0000", 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294281+0000", 2024-04-24T18:08:13.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294282+0000", 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.39e-06 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294286+0000", 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7919999999999999e-06 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294288+0000", 2024-04-24T18:08:13.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9439999999999999e-06 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60058 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.268519+0000", 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.024116829999999999, 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.032782479000000003, 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60058 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268519+0000", 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268519+0000", 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268519+0000", 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268526+0000", 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7120000000000003e-06 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268532+0000", 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7089999999999996e-06 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.268536+0000", 2024-04-24T18:08:13.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0940000000000001e-06 2024-04-24T18:08:13.029 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.029 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60089 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.274197+0000", 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.018439211, 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.027166605, 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60089 2024-04-24T18:08:13.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274197+0000", 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274197+0000", 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274197+0000", 2024-04-24T18:08:13.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274198+0000", 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4500000000000001e-06 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274202+0000", 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.76e-06 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.274204+0000", 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0849999999999999e-06 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60187 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.280822+0000", 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.011814147000000001, 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.020609364000000002, 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.033 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60187 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280822+0000", 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280822+0000", 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280822+0000", 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280824+0000", 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280830+0000", 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8050000000000003e-06 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.280833+0000", 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.72e-06 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60315 4.19 4.81ac95d9 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.287712+0000", 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.004923691, 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.013784104, 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.036 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60315 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287712+0000", 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287712+0000", 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287712+0000", 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287714+0000", 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4249999999999999e-06 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287717+0000", 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9780000000000001e-06 2024-04-24T18:08:13.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.287719+0000", 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.03e-06 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60379 4.11 4.b72d4db1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.290686+0000", 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.00194988, 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.010854979000000001, 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60379 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290686+0000", 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290686+0000", 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290686+0000", 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290688+0000", 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2620000000000001e-06 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290695+0000", 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3400000000000003e-06 2024-04-24T18:08:13.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.290698+0000", 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.715e-06 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:60443 4.11 4.c5ac20d1 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.294290+0000", 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4294967295.9983459, 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0072859600000000002, 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:13.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 60443 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294290+0000", 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294290+0000", 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:13.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294290+0000", 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294294+0000", 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4100000000000001e-06 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294298+0000", 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4340000000000001e-06 2024-04-24T18:08:13.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.294300+0000", 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 221 2024-04-24T18:08:13.045 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:13.046 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:13.047 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:13.052 INFO:teuthology.orchestra.run.smithi049.stdout:2462 read 0x47d0c15 thru 0x47d8bb3 (0x7f9f bytes) 2024-04-24T18:08:13.052 INFO:teuthology.orchestra.run.smithi049.stdout:2463 write 0xa8b1b38 thru 0xa8bc29d (0xa766 bytes) 2024-04-24T18:08:13.052 INFO:teuthology.orchestra.run.smithi049.stdout:2464 punch from 0x9bed118 to 0x9bf7524, (0xa40c bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2465 write 0xed8fcaf thru 0xed9939e (0x96f0 bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2466 write 0x6127900 thru 0x6132bb4 (0xb2b5 bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2467 read 0x88c29e2 thru 0x88ce55f (0xbb7e bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2468 writesame 0xb464c4c thru 0xb471ad7 data_size 0x2(0xce8c bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2469 writesame 0xe157f8a thru 0xe163e11 data_size 0x86(0xbe88 bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2470 write 0x6e21e8c thru 0x6e22db9 (0xf2e bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2471 trunc from 0xed9939f to 0x6126b3 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2475 trunc from 0x6126b3 to 0x7858a72 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2476 punch from 0x5784305 to 0x5785291, (0xf8c bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2477 read 0x3aebe2e thru 0x3aece65 (0x1038 bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2479 punch from 0x4694e99 to 0x469fd23, (0xae8a bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2481 write 0x6d31a7f thru 0x6d3b330 (0x98b2 bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2483 punch from 0x291f07c to 0x2922104, (0x3088 bytes) 2024-04-24T18:08:13.053 INFO:teuthology.orchestra.run.smithi049.stdout:2485 punch from 0xae04e8 to 0xae324a, (0x2d62 bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2487 punch from 0x84f823 to 0x8541c4, (0x49a1 bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2488 read 0x6531055 thru 0x6531115 (0xc1 bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2491 write 0x2fd250a thru 0x2fd38dc (0x13d3 bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2496 writesame 0xc257935 thru 0xc266741 data_size 0x1(0xee0d bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2498 trunc from 0xc266742 to 0x20086bf 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2499 read 0xfc9542 thru 0xfcbfca (0x2a89 bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2500 write 0x425b60e thru 0x4260838 (0x522b bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2501 write 0x823440e thru 0x823f616 (0xb209 bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2504 writesame 0xd725d88 thru 0xd73102b data_size 0x4a(0xb2a4 bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2509 writesame 0xb184817 thru 0xb189d22 data_size 0x2(0x550c bytes) 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.054 INFO:teuthology.orchestra.run.smithi049.stdout:2511 write 0x531c5a1 thru 0x532b504 (0xef64 bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2513 write 0xedd3476 thru 0xeddf79c (0xc327 bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2514 writesame 0xbedbe27 thru 0xbedeab1 data_size 0x15(0x2c8b bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2515 read 0xd974d07 thru 0xd976b33 (0x1e2d bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2516 read 0x268f71b thru 0x2691a19 (0x22ff bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2519 writesame 0xba7d894 thru 0xba8382a data_size 0x3(0x5f97 bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2524 read 0x2304ce1 thru 0x2308017 (0x3337 bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2525 writesame 0x13e1627 thru 0x13e8c21 data_size 0x1(0x75fb bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2526 read 0x265b4b1 thru 0x266a405 (0xef55 bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2528 writesame 0x2f85c80 thru 0x2f8ea80 data_size 0x1(0x8e01 bytes) 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2531 trunc from 0xeddf79d to 0x479884c 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.055 INFO:teuthology.orchestra.run.smithi049.stdout:2533 write 0x377f23e thru 0x378483a (0x55fd bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2535 write 0x60fc530 thru 0x60fcfd8 (0xaa9 bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2537 trunc from 0x60fcfd9 to 0xcba0f6d 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2539 writesame 0x8e7a2b2 thru 0x8e808dc data_size 0x1(0x662b bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2543 write 0x9a5e021 thru 0x9a635d4 (0x55b4 bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2545 trunc from 0xcba0f6d to 0x8c55658 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2547 write 0x656e4ed thru 0x6575b9a (0x76ae bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2548 read 0x2f31a69 thru 0x2f4080e (0xeda6 bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2550 write 0x4eafa12 thru 0x4eb4c4b (0x523a bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2551 writesame 0x145d010 thru 0x146c3ba data_size 0x57(0xf3ab bytes) 2024-04-24T18:08:13.056 INFO:teuthology.orchestra.run.smithi049.stdout:2552 write 0xa6212e1 thru 0xa629415 (0x8135 bytes) 2024-04-24T18:08:13.057 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.7 to 1 (10000) 2024-04-24T18:08:13.057 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:08:13.057 INFO:teuthology.orchestra.run.smithi049.stdout:2553 write 0x4eddacb thru 0x4ee2dd7 (0x530d bytes) 2024-04-24T18:08:13.057 INFO:teuthology.orchestra.run.smithi049.stdout:2554 read 0x306ecb3 thru 0x307be28 (0xd176 bytes) 2024-04-24T18:08:13.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-24T18:08:13.059 INFO:teuthology.orchestra.run.smithi049.stdout:2556 read 0x10ca23a thru 0x10d531c (0xb0e3 bytes) 2024-04-24T18:08:13.059 INFO:teuthology.orchestra.run.smithi049.stdout:2559 trunc from 0xa629416 to 0xfe0866 2024-04-24T18:08:13.059 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.059 INFO:teuthology.orchestra.run.smithi049.stdout:2561 clone 21 order 18 su 4096 sc 2 2024-04-24T18:08:13.060 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:13.055+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:13.148 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:13.156+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:13.230 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:13.231 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:13.231 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:13.242 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:13.249 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:13.257+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:13.251 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:13.271 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:13.299 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:13.319 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:13.340 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:13.349 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:13.358+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:13.359 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:13.378 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:13.381 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:13.381 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:13.392 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:13.397 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:13.399 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:13.451 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:13.460+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:13.531 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:13.531 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:13.541 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:13.585 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:13.593+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:13.651 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:13.659+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:13.665 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:13.665 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:13.675 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:13.695 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.1 to 1 (10000) 2024-04-24T18:08:13.707 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-24T18:08:13.707 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:08:13.760 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:13.768+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:13.798 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #19, image image_client.0-clone19 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent20 2024-04-24T18:08:13.825 INFO:teuthology.orchestra.run.smithi049.stdout:2562 write 0xb8e3fcc thru 0xb8f11e3 (0xd218 bytes) 2024-04-24T18:08:13.828 INFO:teuthology.orchestra.run.smithi049.stdout:2563 read 0xab02ceb thru 0xab11a5f (0xed75 bytes) 2024-04-24T18:08:13.829 INFO:teuthology.orchestra.run.smithi049.stdout:2564 writesame 0x3c55883 thru 0x3c55ae7 data_size 0x1(0x265 bytes) 2024-04-24T18:08:13.831 INFO:teuthology.orchestra.run.smithi049.stdout:2565 read 0x139d9d7 thru 0x13a8399 (0xa9c3 bytes) 2024-04-24T18:08:13.832 INFO:teuthology.orchestra.run.smithi049.stdout:2566 read 0x284c7e3 thru 0x284f576 (0x2d94 bytes) 2024-04-24T18:08:13.832 INFO:teuthology.orchestra.run.smithi049.stdout:2569 write 0x4556260 thru 0x4564ab0 (0xe851 bytes) 2024-04-24T18:08:13.839 INFO:teuthology.orchestra.run.smithi049.stdout:2571 punch from 0xb1eda40 to 0xb1f2423, (0x49e3 bytes) 2024-04-24T18:08:13.839 INFO:teuthology.orchestra.run.smithi049.stdout:2573 read 0x49511d7 thru 0x49567f7 (0x5621 bytes) 2024-04-24T18:08:13.839 INFO:teuthology.orchestra.run.smithi049.stdout:2574 trunc from 0xb8f11e4 to 0xbf93501 2024-04-24T18:08:13.844 INFO:teuthology.orchestra.run.smithi049.stdout:2575 read 0x26ac03e thru 0x26b52d5 (0x9298 bytes) 2024-04-24T18:08:13.844 INFO:teuthology.orchestra.run.smithi049.stdout:2576 writesame 0x9995690 thru 0x9995e0b data_size 0x2(0x77c bytes) 2024-04-24T18:08:13.845 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:13.845 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:13.847 INFO:teuthology.orchestra.run.smithi049.stdout:2578 read 0xa90b090 thru 0xa90d358 (0x22c9 bytes) 2024-04-24T18:08:13.848 INFO:teuthology.orchestra.run.smithi049.stdout:2579 punch from 0xa101d4 to 0xa1cdda, (0xcc06 bytes) 2024-04-24T18:08:13.848 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.848 INFO:teuthology.orchestra.run.smithi049.stdout:2581 write 0x4394c31 thru 0x439c8a1 (0x7c71 bytes) 2024-04-24T18:08:13.852 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:13.860+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:13.855 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:13.858 INFO:teuthology.orchestra.run.smithi049.stdout:2582 trunc from 0xbf93501 to 0x8390444 2024-04-24T18:08:13.868 INFO:teuthology.orchestra.run.smithi049.stdout:2583 read 0x55b91cd thru 0x55c5840 (0xc674 bytes) 2024-04-24T18:08:13.868 INFO:teuthology.orchestra.run.smithi049.stdout:2584 writesame 0x62f00e6 thru 0x62f212b data_size 0x22(0x2046 bytes) 2024-04-24T18:08:13.880 INFO:teuthology.orchestra.run.smithi049.stdout:2585 writesame 0x951335a thru 0x951da8b data_size 0x1(0xa732 bytes) 2024-04-24T18:08:13.887 INFO:teuthology.orchestra.run.smithi049.stdout:2586 writesame 0x9be8073 thru 0x9bf5e53 data_size 0x1(0xdde1 bytes) 2024-04-24T18:08:13.887 INFO:teuthology.orchestra.run.smithi049.stdout:2587 punch from 0xf82c57 to 0xf837b6, (0xb5f bytes) 2024-04-24T18:08:13.887 INFO:teuthology.orchestra.run.smithi049.stdout:2589 write 0x7a369d9 thru 0x7a4134b (0xa973 bytes) 2024-04-24T18:08:13.891 INFO:teuthology.orchestra.run.smithi049.stdout:2590 punch from 0x4714ced to 0x4724cd0, (0xffe3 bytes) 2024-04-24T18:08:13.891 INFO:teuthology.orchestra.run.smithi049.stdout:2592 write 0x16d62b1 thru 0x16e3310 (0xd060 bytes) 2024-04-24T18:08:13.901 INFO:teuthology.orchestra.run.smithi049.stdout:2594 write 0xa34ab58 thru 0xa355b37 (0xafe0 bytes) 2024-04-24T18:08:13.905 INFO:teuthology.orchestra.run.smithi049.stdout:2595 punch from 0x309ac3f to 0x309bf28, (0x12e9 bytes) 2024-04-24T18:08:13.905 INFO:teuthology.orchestra.run.smithi049.stdout:2596 read 0x2628869 thru 0x2629792 (0xf2a bytes) 2024-04-24T18:08:13.905 INFO:teuthology.orchestra.run.smithi049.stdout:2597 punch from 0x764dacb to 0x76504ce, (0x2a03 bytes) 2024-04-24T18:08:13.905 INFO:teuthology.orchestra.run.smithi049.stdout:2598 trunc from 0xa355b38 to 0xa06530e 2024-04-24T18:08:13.921 INFO:teuthology.orchestra.run.smithi049.stdout:2600 writesame 0xaf79e5b thru 0xaf84da2 data_size 0x8e(0xaf48 bytes) 2024-04-24T18:08:13.921 INFO:teuthology.orchestra.run.smithi049.stdout:2601 read 0x2128ad2 thru 0x212a067 (0x1596 bytes) 2024-04-24T18:08:13.921 INFO:teuthology.orchestra.run.smithi049.stdout:2602 punch from 0x119eb92 to 0x11ace3c, (0xe2aa bytes) 2024-04-24T18:08:13.927 INFO:teuthology.orchestra.run.smithi049.stdout:2604 trunc from 0xaf84da3 to 0x590953b 2024-04-24T18:08:13.937 INFO:teuthology.orchestra.run.smithi049.stdout:2606 read 0xbe8cb9 thru 0xbeb42b (0x2773 bytes) 2024-04-24T18:08:13.937 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.937 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.937 INFO:teuthology.orchestra.run.smithi049.stdout:2611 punch from 0x3eec884 to 0x3ef74a0, (0xac1c bytes) 2024-04-24T18:08:13.937 INFO:teuthology.orchestra.run.smithi049.stdout:2613 writesame 0x2daac15 thru 0x2db6595 data_size 0xd(0xb981 bytes) 2024-04-24T18:08:13.941 INFO:teuthology.orchestra.run.smithi049.stdout:2614 write 0x417959f thru 0x41799e8 (0x44a bytes) 2024-04-24T18:08:13.945 INFO:teuthology.orchestra.run.smithi049.stdout:2618 read 0x14531fc thru 0x1460632 (0xd437 bytes) 2024-04-24T18:08:13.946 INFO:teuthology.orchestra.run.smithi049.stdout:2619 trunc from 0x590953b to 0x47a21cf 2024-04-24T18:08:13.951 INFO:teuthology.orchestra.run.smithi049.stdout:2620 write 0x30e784c thru 0x30f5de4 (0xe599 bytes) 2024-04-24T18:08:13.952 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:13.961+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:13.957 INFO:teuthology.orchestra.run.smithi049.stdout:2622 read 0x414cff thru 0x41904b (0x434d bytes) 2024-04-24T18:08:13.959 INFO:teuthology.orchestra.run.smithi049.stdout:2623 trunc from 0x47a21cf to 0x6428f24 2024-04-24T18:08:13.964 INFO:teuthology.orchestra.run.smithi049.stdout:2625 read 0x258735d thru 0x2590dcc (0x9a70 bytes) 2024-04-24T18:08:13.974 INFO:teuthology.orchestra.run.smithi049.stdout:2627 trunc from 0x6428f24 to 0xe8fa0c5 2024-04-24T18:08:13.980 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.980 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:13.980 INFO:teuthology.orchestra.run.smithi049.stdout:2632 writesame 0x86fccf8 thru 0x8706bef data_size 0x4(0x9ef8 bytes) 2024-04-24T18:08:13.987 INFO:teuthology.orchestra.run.smithi049.stdout:2633 read 0xc2ca376 thru 0xc2cbace (0x1759 bytes) 2024-04-24T18:08:13.987 INFO:teuthology.orchestra.run.smithi049.stdout:2634 write 0x9e5f087 thru 0x9e6b718 (0xc692 bytes) 2024-04-24T18:08:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:2635 read 0x9376f4a thru 0x937ee44 (0x7efb bytes) 2024-04-24T18:08:13.991 INFO:teuthology.orchestra.run.smithi049.stdout:2638 write 0x6694792 thru 0x6697ae7 (0x3356 bytes) 2024-04-24T18:08:13.995 INFO:teuthology.orchestra.run.smithi049.stdout:2639 read 0xe18cdea thru 0xe195297 (0x84ae bytes) 2024-04-24T18:08:13.995 INFO:teuthology.orchestra.run.smithi049.stdout:2640 writesame 0x802dd6 thru 0x8030fc data_size 0x1(0x327 bytes) 2024-04-24T18:08:13.999 INFO:teuthology.orchestra.run.smithi049.stdout:2641 writesame 0x7d7ec68 thru 0x7d84d94 data_size 0x1(0x612d bytes) 2024-04-24T18:08:14.011 INFO:teuthology.orchestra.run.smithi049.stdout:2642 write 0x5af3be6 thru 0x5af6bfa (0x3015 bytes) 2024-04-24T18:08:14.015 INFO:teuthology.orchestra.run.smithi049.stdout:2643 read 0x461c6d3 thru 0x4622063 (0x5991 bytes) 2024-04-24T18:08:14.015 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2646 punch from 0xa260959 to 0xa26143d, (0xae4 bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2648 punch from 0x25bcdb7 to 0x25c5915, (0x8b5e bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2649 read 0xc4f9976 thru 0xc4fa6ca (0xd55 bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2652 read 0x62cce30 thru 0x62d4ca5 (0x7e76 bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2655 punch from 0x80f1841 to 0x80ff75c, (0xdf1b bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2656 punch from 0x9a77f57 to 0x9a7a9ea, (0x2a93 bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2657 punch from 0x56eec52 to 0x56fbded, (0xd19b bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2659 punch from 0x847406 to 0x853e62, (0xca5c bytes) 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.016 INFO:teuthology.orchestra.run.smithi049.stdout:2661 write 0xab2a7ca thru 0xab3800b (0xd842 bytes) 2024-04-24T18:08:14.024 INFO:teuthology.orchestra.run.smithi049.stdout:2662 trunc from 0xe8fa0c5 to 0xd0a47a2 2024-04-24T18:08:14.030 INFO:teuthology.orchestra.run.smithi049.stdout:2663 writesame 0x43ccca9 thru 0x43d7dfb data_size 0x7(0xb153 bytes) 2024-04-24T18:08:14.030 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:14.030 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:14.033 INFO:teuthology.orchestra.run.smithi049.stdout:2664 write 0x83e8f90 thru 0x83f047e (0x74ef bytes) 2024-04-24T18:08:14.039 INFO:teuthology.orchestra.run.smithi049.stdout:2665 punch from 0x92d3fbf to 0x92e1353, (0xd394 bytes) 2024-04-24T18:08:14.039 INFO:teuthology.orchestra.run.smithi049.stdout:2666 read 0xde7f88 thru 0xdf3bbc (0xbc35 bytes) 2024-04-24T18:08:14.039 INFO:teuthology.orchestra.run.smithi049.stdout:2667 writesame 0x905f6b3 thru 0x906bc26 data_size 0x2(0xc574 bytes) 2024-04-24T18:08:14.041 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:14.050 INFO:teuthology.orchestra.run.smithi049.stdout:2668 writesame 0x9dd3da2 thru 0x9dddc6b data_size 0x32(0x9eca bytes) 2024-04-24T18:08:14.053 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:14.062+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:14.055 INFO:teuthology.orchestra.run.smithi049.stdout:2669 write 0x4faea75 thru 0x4fb28eb (0x3e77 bytes) 2024-04-24T18:08:14.059 INFO:teuthology.orchestra.run.smithi049.stdout:2670 write 0x52e56f6 thru 0x52e9790 (0x409b bytes) 2024-04-24T18:08:14.070 INFO:teuthology.orchestra.run.smithi049.stdout:2671 write 0xe05fadc thru 0xe06d716 (0xdc3b bytes) 2024-04-24T18:08:14.080 INFO:teuthology.orchestra.run.smithi049.stdout:2673 trunc from 0xe06d717 to 0x885185a 2024-04-24T18:08:14.093 INFO:teuthology.orchestra.run.smithi049.stdout:2674 trunc from 0x885185a to 0xacf6ede 2024-04-24T18:08:14.099 INFO:teuthology.orchestra.run.smithi049.stdout:2675 trunc from 0xacf6ede to 0xb9c7339 2024-04-24T18:08:14.111 INFO:teuthology.orchestra.run.smithi049.stdout:2676 writesame 0xc3ef141 thru 0xc3fe8bd data_size 0x31(0xf77d bytes) 2024-04-24T18:08:14.111 INFO:teuthology.orchestra.run.smithi049.stdout:2677 read 0x730f17b thru 0x731d320 (0xe1a6 bytes) 2024-04-24T18:08:14.111 INFO:teuthology.orchestra.run.smithi049.stdout:2678 read 0x28af844 thru 0x28bf166 (0xf923 bytes) 2024-04-24T18:08:14.111 INFO:teuthology.orchestra.run.smithi049.stdout:2682 write 0x32f585e thru 0x32f8342 (0x2ae5 bytes) 2024-04-24T18:08:14.118 INFO:teuthology.orchestra.run.smithi049.stdout:2683 trunc from 0xc3fe8be to 0xacf9393 2024-04-24T18:08:14.132 INFO:teuthology.orchestra.run.smithi049.stdout:2684 write 0xdffc5c1 thru 0xe003392 (0x6dd2 bytes) 2024-04-24T18:08:14.140 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:08:14.141 INFO:teuthology.orchestra.run.smithi049.stdout:2685 read 0x10d015f thru 0x10dd5c7 (0xd469 bytes) 2024-04-24T18:08:14.141 INFO:teuthology.orchestra.run.smithi049.stdout:2686 writesame 0x39a75d2 thru 0x39b5be4 data_size 0x1d(0xe613 bytes) 2024-04-24T18:08:14.142 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:08:14.148 INFO:teuthology.orchestra.run.smithi049.stdout:2687 writesame 0xa5e0822 thru 0xa5e1aab data_size 0x15(0x128a bytes) 2024-04-24T18:08:14.154 INFO:teuthology.orchestra.run.smithi049.stdout:2688 punch from 0x9fd778a to 0x9fe3df3, (0xc669 bytes) 2024-04-24T18:08:14.154 INFO:teuthology.orchestra.run.smithi049.stdout:2689 write 0x76242d5 thru 0x762ee6f (0xab9b bytes) 2024-04-24T18:08:14.155 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:14.162+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:14.156 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-04-24T18:08:13.698432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":573319581,"num_objects":816,"num_object_clones":270,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":816,"num_whiteouts":247,"num_read":78416,"num_read_kb":244670,"num_write":8717,"num_write_kb":140027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":18486782,"num_keys_recovered":80,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6631,"ondisk_log_size":6631,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2463796,"kb_used_data":35680,"kb_used_omap":13,"kb_used_meta":2427890,"kb_avail":752510924,"statfs":{"total":773094113280,"available":770571186176,"internally_reserved":0,"allocated":36536320,"data_stored":276930900,"data_compressed":6302337,"data_compressed_allocated":17985536,"data_compressed_original":263246956,"omap_allocated":13632,"internal_metadata":2486160064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":136496788,"num_objects":149,"num_object_clones":48,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":40,"num_read":15078,"num_read_kb":44112,"num_write":1611,"num_write_kb":27211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023593"},"pg_stats":[{"pgid":"4.1f","version":"60'155","reported_seq":1651,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.902994+0000","last_change":"2024-04-24T18:07:38.288543+0000","last_active":"2024-04-24T18:08:11.902994+0000","last_peered":"2024-04-24T18:08:11.902994+0000","last_clean":"2024-04-24T18:08:11.902994+0000","last_became_active":"2024-04-24T18:07:36.550017+0000","last_became_peered":"2024-04-24T18:07:36.550017+0000","last_unstale":"2024-04-24T18:08:11.902994+0000","last_undegraded":"2024-04-24T18:08:11.902994+0000","last_fullsized":"2024-04-24T18:08:11.902994+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:32:03.728352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":24272994,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":2044,"num_read_kb":11136,"num_write":208,"num_write_kb":3715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"60'653","reported_seq":1697,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.899969+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:08:11.899969+0000","last_peered":"2024-04-24T18:08:11.899969+0000","last_clean":"2024-04-24T18:08:11.899969+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:08:11.899969+0000","last_undegraded":"2024-04-24T18:08:11.899969+0000","last_fullsized":"2024-04-24T18:08:11.899969+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":17882416,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":4359,"num_read_kb":6852,"num_write":1131,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"60'83","reported_seq":282,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.098232+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:08:12.098232+0000","last_peered":"2024-04-24T18:08:12.098232+0000","last_clean":"2024-04-24T18:08:12.098232+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:08:12.098232+0000","last_undegraded":"2024-04-24T18:08:12.098232+0000","last_fullsized":"2024-04-24T18:08:12.098232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":6422982,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":154,"num_read_kb":3816,"num_write":104,"num_write_kb":3770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"60'306","reported_seq":3525,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.267848+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:08:12.267848+0000","last_peered":"2024-04-24T18:08:12.267848+0000","last_clean":"2024-04-24T18:08:12.267848+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:08:12.267848+0000","last_undegraded":"2024-04-24T18:08:12.267848+0000","last_fullsized":"2024-04-24T18:08:12.267848+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":21248717,"num_objects":29,"num_object_clones":11,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":4194,"num_read_kb":7481,"num_write":447,"num_write_kb":5566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"60'148","reported_seq":2954,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.098656+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:08:12.098656+0000","last_peered":"2024-04-24T18:08:12.098656+0000","last_clean":"2024-04-24T18:08:12.098656+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:08:12.098656+0000","last_undegraded":"2024-04-24T18:08:12.098656+0000","last_fullsized":"2024-04-24T18:08:12.098656+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":32534140,"num_objects":35,"num_object_clones":14,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":2837,"num_read_kb":19154,"num_write":185,"num_write_kb":11787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"60'156","reported_seq":4316,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.352281+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:08:12.352281+0000","last_peered":"2024-04-24T18:08:12.352281+0000","last_clean":"2024-04-24T18:08:12.352281+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:08:12.352281+0000","last_undegraded":"2024-04-24T18:08:12.352281+0000","last_fullsized":"2024-04-24T18:08:12.352281+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":15784183,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":4195,"num_read_kb":11552,"num_write":158,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"60'181","reported_seq":3563,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.330705+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:08:12.330705+0000","last_peered":"2024-04-24T18:08:12.330705+0000","last_clean":"2024-04-24T18:08:12.330705+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:08:12.330705+0000","last_undegraded":"2024-04-24T18:08:12.330705+0000","last_fullsized":"2024-04-24T18:08:12.330705+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":13595601,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":4108,"num_read_kb":12022,"num_write":192,"num_write_kb":2678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'129","reported_seq":3836,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.192105+0000","last_change":"2024-04-24T18:07:38.291282+0000","last_active":"2024-04-24T18:08:12.192105+0000","last_peered":"2024-04-24T18:08:12.192105+0000","last_clean":"2024-04-24T18:08:12.192105+0000","last_became_active":"2024-04-24T18:07:36.548455+0000","last_became_peered":"2024-04-24T18:07:36.548455+0000","last_unstale":"2024-04-24T18:08:12.192105+0000","last_undegraded":"2024-04-24T18:08:12.192105+0000","last_fullsized":"2024-04-24T18:08:12.192105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:50.006764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":28341504,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":3722,"num_read_kb":14520,"num_write":154,"num_write_kb":8652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"60'145","reported_seq":722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.022776+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:08:12.022776+0000","last_peered":"2024-04-24T18:08:12.022776+0000","last_clean":"2024-04-24T18:08:12.022776+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:08:12.022776+0000","last_undegraded":"2024-04-24T18:08:12.022776+0000","last_fullsized":"2024-04-24T18:08:12.022776+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":21696936,"num_objects":25,"num_object_clones":11,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":597,"num_read_kb":5928,"num_write":169,"num_write_kb":5994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"60'320","reported_seq":362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.371465+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:08:12.371465+0000","last_peered":"2024-04-24T18:08:12.371465+0000","last_clean":"2024-04-24T18:08:12.371465+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:08:12.371465+0000","last_undegraded":"2024-04-24T18:08:12.371465+0000","last_fullsized":"2024-04-24T18:08:12.371465+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":14106787,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1503,"num_read_kb":2470,"num_write":531,"num_write_kb":3210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.742376+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:08:10.742376+0000","last_peered":"2024-04-24T18:08:10.742376+0000","last_clean":"2024-04-24T18:08:10.742376+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:08:10.742376+0000","last_undegraded":"2024-04-24T18:08:10.742376+0000","last_fullsized":"2024-04-24T18:08:10.742376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.742378+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:08:10.742378+0000","last_peered":"2024-04-24T18:08:10.742378+0000","last_clean":"2024-04-24T18:08:10.742378+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:08:10.742378+0000","last_undegraded":"2024-04-24T18:08:10.742378+0000","last_fullsized":"2024-04-24T18:08:10.742378+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"60'187","reported_seq":2665,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.358477+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:08:12.358477+0000","last_peered":"2024-04-24T18:08:12.358477+0000","last_clean":"2024-04-24T18:08:12.358477+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:08:12.358477+0000","last_undegraded":"2024-04-24T18:08:12.358477+0000","last_fullsized":"2024-04-24T18:08:12.358477+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":34637299,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2757,"num_read_kb":8030,"num_write":217,"num_write_kb":4425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.679493+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:10.679493+0000","last_peered":"2024-04-24T18:08:10.679493+0000","last_clean":"2024-04-24T18:08:10.679493+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:10.679493+0000","last_undegraded":"2024-04-24T18:08:10.679493+0000","last_fullsized":"2024-04-24T18:08:10.679493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.687899+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:10.687899+0000","last_peered":"2024-04-24T18:08:10.687899+0000","last_clean":"2024-04-24T18:08:10.687899+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:10.687899+0000","last_undegraded":"2024-04-24T18:08:10.687899+0000","last_fullsized":"2024-04-24T18:08:10.687899+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"60'156","reported_seq":382,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.900089+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:08:11.900089+0000","last_peered":"2024-04-24T18:08:11.900089+0000","last_clean":"2024-04-24T18:08:11.900089+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:08:11.900089+0000","last_undegraded":"2024-04-24T18:08:11.900089+0000","last_fullsized":"2024-04-24T18:08:11.900089+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":3622601,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":828,"num_read_kb":903,"num_write":174,"num_write_kb":806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.682967+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:10.682967+0000","last_peered":"2024-04-24T18:08:10.682967+0000","last_clean":"2024-04-24T18:08:10.682967+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:10.682967+0000","last_undegraded":"2024-04-24T18:08:10.682967+0000","last_fullsized":"2024-04-24T18:08:10.682967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:13:26.133189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.772020+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:10.772020+0000","last_peered":"2024-04-24T18:08:10.772020+0000","last_clean":"2024-04-24T18:08:10.772020+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:10.772020+0000","last_undegraded":"2024-04-24T18:08:10.772020+0000","last_fullsized":"2024-04-24T18:08:10.772020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"60'441","reported_seq":3916,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.901957+0000","last_change":"2024-04-24T18:07:38.288298+0000","last_active":"2024-04-24T18:08:11.901957+0000","last_peered":"2024-04-24T18:08:11.901957+0000","last_clean":"2024-04-24T18:08:11.901957+0000","last_became_active":"2024-04-24T18:07:36.549561+0000","last_became_peered":"2024-04-24T18:07:36.549561+0000","last_unstale":"2024-04-24T18:08:11.901957+0000","last_undegraded":"2024-04-24T18:08:11.901957+0000","last_fullsized":"2024-04-24T18:08:11.901957+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:48.800593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":17685506,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":6343,"num_read_kb":16293,"num_write":603,"num_write_kb":4922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.677346+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:10.677346+0000","last_peered":"2024-04-24T18:08:10.677346+0000","last_clean":"2024-04-24T18:08:10.677346+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:10.677346+0000","last_undegraded":"2024-04-24T18:08:10.677346+0000","last_fullsized":"2024-04-24T18:08:10.677346+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.771785+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:10.771785+0000","last_peered":"2024-04-24T18:08:10.771785+0000","last_clean":"2024-04-24T18:08:10.771785+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:10.771785+0000","last_undegraded":"2024-04-24T18:08:10.771785+0000","last_fullsized":"2024-04-24T18:08:10.771785+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"59'131","reported_seq":411,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.842369+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:08:11.842369+0000","last_peered":"2024-04-24T18:08:11.842369+0000","last_clean":"2024-04-24T18:08:11.842369+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:08:11.842369+0000","last_undegraded":"2024-04-24T18:08:11.842369+0000","last_fullsized":"2024-04-24T18:08:11.842369+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":7706005,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":1799,"num_read_kb":6102,"num_write":284,"num_write_kb":2979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.677536+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:10.677536+0000","last_peered":"2024-04-24T18:08:10.677536+0000","last_clean":"2024-04-24T18:08:10.677536+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:10.677536+0000","last_undegraded":"2024-04-24T18:08:10.677536+0000","last_fullsized":"2024-04-24T18:08:10.677536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:25.729450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":249,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.772030+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:10.772030+0000","last_peered":"2024-04-24T18:08:10.772030+0000","last_clean":"2024-04-24T18:08:10.772030+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:10.772030+0000","last_undegraded":"2024-04-24T18:08:10.772030+0000","last_fullsized":"2024-04-24T18:08:10.772030+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.677448+0000","last_change":"2024-04-24T18:07:56.624753+0000","last_active":"2024-04-24T18:08:10.677448+0000","last_peered":"2024-04-24T18:08:10.677448+0000","last_clean":"2024-04-24T18:08:10.677448+0000","last_became_active":"2024-04-24T18:07:56.624464+0000","last_became_peered":"2024-04-24T18:07:56.624464+0000","last_unstale":"2024-04-24T18:08:10.677448+0000","last_undegraded":"2024-04-24T18:08:10.677448+0000","last_fullsized":"2024-04-24T18:08:10.677448+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:06:20.390454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"60'209","reported_seq":4758,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.098736+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:08:12.098736+0000","last_peered":"2024-04-24T18:08:12.098736+0000","last_clean":"2024-04-24T18:08:12.098736+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:08:12.098736+0000","last_undegraded":"2024-04-24T18:08:12.098736+0000","last_fullsized":"2024-04-24T18:08:12.098736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":19002662,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":5801,"num_read_kb":10797,"num_write":215,"num_write_kb":3577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"59'87","reported_seq":307,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.310132+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:08:11.310132+0000","last_peered":"2024-04-24T18:08:11.310132+0000","last_clean":"2024-04-24T18:08:11.310132+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:08:11.310132+0000","last_undegraded":"2024-04-24T18:08:11.310132+0000","last_fullsized":"2024-04-24T18:08:11.310132+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":10020979,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":248,"num_read_kb":643,"num_write":85,"num_write_kb":1522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":52,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.683172+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:10.683172+0000","last_peered":"2024-04-24T18:08:10.683172+0000","last_clean":"2024-04-24T18:08:10.683172+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:10.683172+0000","last_undegraded":"2024-04-24T18:08:10.683172+0000","last_fullsized":"2024-04-24T18:08:10.683172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"60'138","reported_seq":1552,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.726933+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:08:11.726933+0000","last_peered":"2024-04-24T18:08:11.726933+0000","last_clean":"2024-04-24T18:08:11.726933+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:08:11.726933+0000","last_undegraded":"2024-04-24T18:08:11.726933+0000","last_fullsized":"2024-04-24T18:08:11.726933+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":20016101,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":1481,"num_read_kb":4572,"num_write":143,"num_write_kb":1191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.679640+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:10.679640+0000","last_peered":"2024-04-24T18:08:10.679640+0000","last_clean":"2024-04-24T18:08:10.679640+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:10.679640+0000","last_undegraded":"2024-04-24T18:08:10.679640+0000","last_fullsized":"2024-04-24T18:08:10.679640+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"60'133","reported_seq":3090,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.262385+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:08:12.262385+0000","last_peered":"2024-04-24T18:08:12.262385+0000","last_clean":"2024-04-24T18:08:12.262385+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:08:12.262385+0000","last_undegraded":"2024-04-24T18:08:12.262385+0000","last_fullsized":"2024-04-24T18:08:12.262385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":14396879,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":3460,"num_read_kb":4765,"num_write":121,"num_write_kb":2561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'138","reported_seq":1799,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.875104+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:08:11.875104+0000","last_peered":"2024-04-24T18:08:11.875104+0000","last_clean":"2024-04-24T18:08:11.875104+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:08:11.875104+0000","last_undegraded":"2024-04-24T18:08:11.875104+0000","last_fullsized":"2024-04-24T18:08:11.875104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":30511871,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":1814,"num_read_kb":7162,"num_write":182,"num_write_kb":4115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"60'278","reported_seq":747,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.907029+0000","last_change":"2024-04-24T18:07:56.696872+0000","last_active":"2024-04-24T18:08:11.907029+0000","last_peered":"2024-04-24T18:08:11.907029+0000","last_clean":"2024-04-24T18:08:11.907029+0000","last_became_active":"2024-04-24T18:07:56.627442+0000","last_became_peered":"2024-04-24T18:07:56.627442+0000","last_unstale":"2024-04-24T18:08:11.907029+0000","last_undegraded":"2024-04-24T18:08:11.907029+0000","last_fullsized":"2024-04-24T18:08:11.907029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:35.221342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":10478912,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":1294,"num_read_kb":1330,"num_write":403,"num_write_kb":2498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1623115,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"60'91","reported_seq":917,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.897768+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:08:11.897768+0000","last_peered":"2024-04-24T18:08:11.897768+0000","last_clean":"2024-04-24T18:08:11.897768+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:08:11.897768+0000","last_undegraded":"2024-04-24T18:08:11.897768+0000","last_fullsized":"2024-04-24T18:08:11.897768+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":9814362,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":769,"num_read_kb":1536,"num_write":90,"num_write_kb":2357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"60'134","reported_seq":572,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.024177+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:08:12.024177+0000","last_peered":"2024-04-24T18:08:12.024177+0000","last_clean":"2024-04-24T18:08:12.024177+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:08:12.024177+0000","last_undegraded":"2024-04-24T18:08:12.024177+0000","last_fullsized":"2024-04-24T18:08:12.024177+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":14774119,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":678,"num_read_kb":3431,"num_write":173,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"60'134","reported_seq":698,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.989713+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:08:11.989713+0000","last_peered":"2024-04-24T18:08:11.989713+0000","last_clean":"2024-04-24T18:08:11.989713+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:08:11.989713+0000","last_undegraded":"2024-04-24T18:08:11.989713+0000","last_fullsized":"2024-04-24T18:08:11.989713+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":23278994,"num_objects":36,"num_object_clones":13,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":835,"num_read_kb":1995,"num_write":116,"num_write_kb":1358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"60'181","reported_seq":1107,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.904597+0000","last_change":"2024-04-24T18:07:38.291549+0000","last_active":"2024-04-24T18:08:11.904597+0000","last_peered":"2024-04-24T18:08:11.904597+0000","last_clean":"2024-04-24T18:08:11.904597+0000","last_became_active":"2024-04-24T18:07:36.549997+0000","last_became_peered":"2024-04-24T18:07:36.549997+0000","last_unstale":"2024-04-24T18:08:11.904597+0000","last_undegraded":"2024-04-24T18:08:11.904597+0000","last_fullsized":"2024-04-24T18:08:11.904597+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:15.050671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":7170363,"num_objects":27,"num_object_clones":11,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":1610,"num_read_kb":3597,"num_write":186,"num_write_kb":2745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'367","reported_seq":1796,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.282928+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:08:12.282928+0000","last_peered":"2024-04-24T18:08:12.282928+0000","last_clean":"2024-04-24T18:08:12.282928+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:08:12.282928+0000","last_undegraded":"2024-04-24T18:08:12.282928+0000","last_fullsized":"2024-04-24T18:08:12.282928+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":11111637,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":3627,"num_read_kb":12980,"num_write":514,"num_write_kb":4852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"60'175","reported_seq":1399,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.307767+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:08:12.307767+0000","last_peered":"2024-04-24T18:08:12.307767+0000","last_clean":"2024-04-24T18:08:12.307767+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:08:12.307767+0000","last_undegraded":"2024-04-24T18:08:12.307767+0000","last_fullsized":"2024-04-24T18:08:12.307767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":10865259,"num_objects":34,"num_object_clones":15,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":1824,"num_read_kb":3656,"num_write":166,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"60'186","reported_seq":2510,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.334565+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:08:12.334565+0000","last_peered":"2024-04-24T18:08:12.334565+0000","last_clean":"2024-04-24T18:08:12.334565+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:08:12.334565+0000","last_undegraded":"2024-04-24T18:08:12.334565+0000","last_fullsized":"2024-04-24T18:08:12.334565+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":60915100,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":2940,"num_read_kb":27104,"num_write":238,"num_write_kb":15543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"60'153","reported_seq":495,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.367694+0000","last_change":"2024-04-24T18:07:56.675621+0000","last_active":"2024-04-24T18:08:12.367694+0000","last_peered":"2024-04-24T18:08:12.367694+0000","last_clean":"2024-04-24T18:08:12.367694+0000","last_became_active":"2024-04-24T18:07:56.624554+0000","last_became_peered":"2024-04-24T18:07:56.624554+0000","last_unstale":"2024-04-24T18:08:12.367694+0000","last_undegraded":"2024-04-24T18:08:12.367694+0000","last_fullsized":"2024-04-24T18:08:12.367694+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:23.574241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":11425229,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":2730,"num_read_kb":10870,"num_write":160,"num_write_kb":4156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2974171,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"59'106","reported_seq":2782,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.310192+0000","last_change":"2024-04-24T18:07:38.292302+0000","last_active":"2024-04-24T18:08:11.310192+0000","last_peered":"2024-04-24T18:08:11.310192+0000","last_clean":"2024-04-24T18:08:11.310192+0000","last_became_active":"2024-04-24T18:07:36.549147+0000","last_became_peered":"2024-04-24T18:07:36.549147+0000","last_unstale":"2024-04-24T18:08:11.310192+0000","last_undegraded":"2024-04-24T18:08:11.310192+0000","last_fullsized":"2024-04-24T18:08:11.310192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:02.455780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":6518858,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":2670,"num_read_kb":5251,"num_write":103,"num_write_kb":3637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"60'176","reported_seq":1496,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.304814+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:08:12.304814+0000","last_peered":"2024-04-24T18:08:12.304814+0000","last_clean":"2024-04-24T18:08:12.304814+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:08:12.304814+0000","last_undegraded":"2024-04-24T18:08:12.304814+0000","last_fullsized":"2024-04-24T18:08:12.304814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":22759733,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":2404,"num_read_kb":6278,"num_write":173,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"60'102","reported_seq":1590,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.074220+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:08:12.074220+0000","last_peered":"2024-04-24T18:08:12.074220+0000","last_clean":"2024-04-24T18:08:12.074220+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:08:12.074220+0000","last_undegraded":"2024-04-24T18:08:12.074220+0000","last_fullsized":"2024-04-24T18:08:12.074220+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":4520721,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":1483,"num_read_kb":4350,"num_write":99,"num_write_kb":2389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"60'535","reported_seq":1529,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.906520+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:08:11.906520+0000","last_peered":"2024-04-24T18:08:11.906520+0000","last_clean":"2024-04-24T18:08:11.906520+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:08:11.906520+0000","last_undegraded":"2024-04-24T18:08:11.906520+0000","last_fullsized":"2024-04-24T18:08:11.906520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":25609763,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":3192,"num_read_kb":7994,"num_write":834,"num_write_kb":2229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":572729213,"num_objects":814,"num_object_clones":270,"num_object_copies":1628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":814,"num_whiteouts":247,"num_read":78300,"num_read_kb":244570,"num_write":8558,"num_write_kb":137611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":17306046,"num_keys_recovered":80,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30941184,"data_stored":274932000,"data_compressed":6301427,"data_compressed_allocated":18063360,"data_compressed_original":263935084,"omap_allocated":0,"internal_metadata":0},"log_size":6514,"ondisk_log_size":6514,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305424,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94066416,"statfs":{"total":96636764160,"available":96324009984,"internally_reserved":0,"allocated":4063232,"data_stored":37398578,"data_compressed":556273,"data_compressed_allocated":2166784,"data_compressed_original":36131589,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305156,"kb_used_data":3708,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94066684,"statfs":{"total":96636764160,"available":96324284416,"internally_reserved":0,"allocated":3796992,"data_stored":28361506,"data_compressed":776140,"data_compressed_allocated":2023424,"data_compressed_original":27074229,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312032,"kb_used_data":6488,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94059808,"statfs":{"total":96636764160,"available":96317243392,"internally_reserved":0,"allocated":6643712,"data_stored":40217421,"data_compressed":903457,"data_compressed_allocated":2744320,"data_compressed_original":37391896,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308076,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063764,"statfs":{"total":96636764160,"available":96321294336,"internally_reserved":0,"allocated":2580480,"data_stored":17338668,"data_compressed":365305,"data_compressed_allocated":1060864,"data_compressed_original":16300816,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303232,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068608,"statfs":{"total":96636764160,"available":96326254592,"internally_reserved":0,"allocated":1822720,"data_stored":8100723,"data_compressed":248060,"data_compressed_allocated":651264,"data_compressed_original":7393280,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313156,"kb_used_data":7548,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94058684,"statfs":{"total":96636764160,"available":96316092416,"internally_reserved":0,"allocated":7729152,"data_stored":73541936,"data_compressed":1451418,"data_compressed_allocated":4452352,"data_compressed_original":70828157,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311532,"kb_used_data":5928,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94060308,"statfs":{"total":96636764160,"available":96317755392,"internally_reserved":0,"allocated":6070272,"data_stored":52729575,"data_compressed":1577610,"data_compressed_allocated":3567616,"data_compressed_original":50750275,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305188,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94066652,"statfs":{"total":96636764160,"available":96324251648,"internally_reserved":0,"allocated":3829760,"data_stored":19242493,"data_compressed":424074,"data_compressed_allocated":1318912,"data_compressed_original":17376714,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":17329292,"data_compressed":368327,"data_compressed_allocated":1097728,"data_compressed_original":16603920,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5775360,"data_stored":39896630,"data_compressed":900517,"data_compressed_allocated":2744320,"data_compressed_original":37506584,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1155072,"data_stored":8042106,"data_compressed":250471,"data_compressed_allocated":667648,"data_compressed_original":7655424,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7086080,"data_stored":73520156,"data_compressed":1453948,"data_compressed_allocated":4476928,"data_compressed_original":71106685,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5386240,"data_stored":52412721,"data_compressed":1577677,"data_compressed_allocated":3571712,"data_compressed_original":50758467,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3137536,"data_stored":18936850,"data_compressed":424275,"data_compressed_allocated":1331200,"data_compressed_original":17401290,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3141632,"data_stored":28312954,"data_compressed":778603,"data_compressed_allocated":2043904,"data_compressed_original":27344565,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3309568,"data_stored":36481291,"data_compressed":547609,"data_compressed_allocated":2129920,"data_compressed_original":35558149,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:08:14.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:08:14.160 INFO:teuthology.orchestra.run.smithi049.stdout:2690 read 0x78a4270 thru 0x78a6c82 (0x2a13 bytes) 2024-04-24T18:08:14.160 INFO:teuthology.orchestra.run.smithi049.stdout:2692 punch from 0xaa779fb to 0xaa7da7f, (0x6084 bytes) 2024-04-24T18:08:14.160 INFO:teuthology.orchestra.run.smithi049.stdout:2693 read 0xa7c54c6 thru 0xa7d395e (0xe499 bytes) 2024-04-24T18:08:14.160 INFO:teuthology.orchestra.run.smithi049.stdout:2695 punch from 0x2466eb0 to 0x247404f, (0xd19f bytes) 2024-04-24T18:08:14.160 INFO:teuthology.orchestra.run.smithi049.stdout:2696 write 0x1d51c98 thru 0x1d610f5 (0xf45e bytes) 2024-04-24T18:08:14.163 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.163 INFO:teuthology.orchestra.run.smithi049.stdout:2698 write 0xc668c69 thru 0xc676bd1 (0xdf69 bytes) 2024-04-24T18:08:14.166 INFO:teuthology.orchestra.run.smithi049.stdout:2699 read 0x58b7d8d thru 0x58bfd7d (0x7ff1 bytes) 2024-04-24T18:08:14.166 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.167 INFO:teuthology.orchestra.run.smithi049.stdout:2701 write 0x5493937 thru 0x5498f09 (0x55d3 bytes) 2024-04-24T18:08:14.170 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.170 INFO:teuthology.orchestra.run.smithi049.stdout:2703 read 0x81c5d67 thru 0x81c62eb (0x585 bytes) 2024-04-24T18:08:14.170 INFO:teuthology.orchestra.run.smithi049.stdout:2705 write 0xaeac770 thru 0xaeba60e (0xde9f bytes) 2024-04-24T18:08:14.173 INFO:teuthology.orchestra.run.smithi049.stdout:2706 punch from 0x48ff62c to 0x4903a26, (0x43fa bytes) 2024-04-24T18:08:14.173 INFO:teuthology.orchestra.run.smithi049.stdout:2707 write 0x3d5e601 thru 0x3d67135 (0x8b35 bytes) 2024-04-24T18:08:14.174 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:14.174 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:14.177 INFO:teuthology.orchestra.run.smithi049.stdout:2708 read 0xbee14bb thru 0xbeec9e9 (0xb52f bytes) 2024-04-24T18:08:14.178 INFO:teuthology.orchestra.run.smithi049.stdout:2711 write 0x92129f thru 0x930b32 (0xf894 bytes) 2024-04-24T18:08:14.182 INFO:teuthology.orchestra.run.smithi049.stdout:2712 read 0x8c2f4c6 thru 0x8c36812 (0x734d bytes) 2024-04-24T18:08:14.183 INFO:teuthology.orchestra.run.smithi049.stdout:2713 write 0x8e0fc9b thru 0x8e18360 (0x86c6 bytes) 2024-04-24T18:08:14.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:14.188 INFO:teuthology.orchestra.run.smithi049.stdout:2714 write 0x7b1d812 thru 0x7b2aa8d (0xd27c bytes) 2024-04-24T18:08:14.193 INFO:teuthology.orchestra.run.smithi049.stdout:2716 punch from 0x7f2a26b to 0x7f32112, (0x7ea7 bytes) 2024-04-24T18:08:14.193 INFO:teuthology.orchestra.run.smithi049.stdout:2717 trunc from 0xe003393 to 0xdfc4d29 2024-04-24T18:08:14.202 INFO:teuthology.orchestra.run.smithi049.stdout:2718 writesame 0x2b23972 thru 0x2b28eca data_size 0x1(0x5559 bytes) 2024-04-24T18:08:14.210 INFO:teuthology.orchestra.run.smithi049.stdout:2719 punch from 0x67d78b7 to 0x67e0be2, (0x932b bytes) 2024-04-24T18:08:14.210 INFO:teuthology.orchestra.run.smithi049.stdout:2720 writesame 0xaa11c52 thru 0xaa136d4 data_size 0x1(0x1a83 bytes) 2024-04-24T18:08:14.215 INFO:teuthology.orchestra.run.smithi049.stdout:2723 read 0xb8eeb5 thru 0xb941b7 (0x5303 bytes) 2024-04-24T18:08:14.215 INFO:teuthology.orchestra.run.smithi049.stdout:2724 punch from 0x7bbf990 to 0x7bc4f5e, (0x55ce bytes) 2024-04-24T18:08:14.215 INFO:teuthology.orchestra.run.smithi049.stdout:2725 write 0xc9f99fc thru 0xca09929 (0xff2e bytes) 2024-04-24T18:08:14.220 INFO:teuthology.orchestra.run.smithi049.stdout:2726 punch from 0x76df8c5 to 0x76e9f0e, (0xa649 bytes) 2024-04-24T18:08:14.220 INFO:teuthology.orchestra.run.smithi049.stdout:2730 read 0x1870a11 thru 0x187150b (0xafb bytes) 2024-04-24T18:08:14.220 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.220 INFO:teuthology.orchestra.run.smithi049.stdout:2732 read 0x9c29536 thru 0x9c374cf (0xdf9a bytes) 2024-04-24T18:08:14.220 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.220 INFO:teuthology.orchestra.run.smithi049.stdout:2734 read 0x8581f52 thru 0x85905dc (0xe68b bytes) 2024-04-24T18:08:14.220 INFO:teuthology.orchestra.run.smithi049.stdout:2735 write 0x66d9ebf thru 0x66e6a08 (0xcb4a bytes) 2024-04-24T18:08:14.222 INFO:teuthology.orchestra.run.smithi049.stdout:2736 read 0x4b9c9e0 thru 0x4b9f9ff (0x3020 bytes) 2024-04-24T18:08:14.222 INFO:teuthology.orchestra.run.smithi049.stdout:2737 punch from 0x2192f12 to 0x21a175e, (0xe84c bytes) 2024-04-24T18:08:14.222 INFO:teuthology.orchestra.run.smithi049.stdout:2738 punch from 0x29c1bbb to 0x29c7953, (0x5d98 bytes) 2024-04-24T18:08:14.222 INFO:teuthology.orchestra.run.smithi049.stdout:2739 write 0x85c09b8 thru 0x85d05a7 (0xfbf0 bytes) 2024-04-24T18:08:14.227 INFO:teuthology.orchestra.run.smithi049.stdout:2740 writesame 0x783170f thru 0x78400a1 data_size 0x1(0xe993 bytes) 2024-04-24T18:08:14.246 INFO:teuthology.orchestra.run.smithi049.stdout:2742 read 0xd4edd1d thru 0xd4f6a6c (0x8d50 bytes) 2024-04-24T18:08:14.246 INFO:teuthology.orchestra.run.smithi049.stdout:2744 write 0xb73646b thru 0xb73d12a (0x6cc0 bytes) 2024-04-24T18:08:14.252 INFO:teuthology.orchestra.run.smithi049.stdout:2747 read 0x35a408e thru 0x35a8bbd (0x4b30 bytes) 2024-04-24T18:08:14.252 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:14.255 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:14.264+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:14.259 INFO:teuthology.orchestra.run.smithi049.stdout:2749 trunc from 0xdfc4d29 to 0x7ea091e 2024-04-24T18:08:14.275 INFO:teuthology.orchestra.run.smithi049.stdout:2750 trunc from 0x7ea091e to 0x9c9bcb3 2024-04-24T18:08:14.284 INFO:teuthology.orchestra.run.smithi049.stdout:2751 trunc from 0x9c9bcb3 to 0xc5ca884 2024-04-24T18:08:14.289 INFO:teuthology.orchestra.run.smithi049.stdout:2752 read 0xb08f97b thru 0xb096d74 (0x73fa bytes) 2024-04-24T18:08:14.289 INFO:teuthology.orchestra.run.smithi049.stdout:2753 writesame 0x794923 thru 0x7a1298 data_size 0x2(0xc976 bytes) 2024-04-24T18:08:14.303 INFO:teuthology.orchestra.run.smithi049.stdout:2754 read 0x58f5393 thru 0x58f6516 (0x1184 bytes) 2024-04-24T18:08:14.303 INFO:teuthology.orchestra.run.smithi049.stdout:2755 punch from 0xcc2bcd to 0xcc6853, (0x3c86 bytes) 2024-04-24T18:08:14.310 INFO:teuthology.orchestra.run.smithi049.stdout:2756 write 0xceec1f5 thru 0xcefb4ca (0xf2d6 bytes) 2024-04-24T18:08:14.314 INFO:teuthology.orchestra.run.smithi049.stdout:2758 read 0x5b7a2b thru 0x5c4a6a (0xd040 bytes) 2024-04-24T18:08:14.325 INFO:teuthology.orchestra.run.smithi049.stdout:2759 trunc from 0xcefb4cb to 0x33a1128 2024-04-24T18:08:14.350 INFO:teuthology.orchestra.run.smithi049.stdout:2760 trunc from 0x33a1128 to 0xb9cb5be 2024-04-24T18:08:14.354 INFO:teuthology.orchestra.run.smithi049.stdout:2761 clone 22 order 21 su 1048576 sc 15 2024-04-24T18:08:14.357 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:14.365+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:14.358 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:14.368 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:14.457 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:14.466+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:14.524 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:08:14.525 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:08:14.533 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:14.533 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:14.536 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":83,"stamp":"2024-04-24T18:08:13.698432+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":573319581,"num_objects":816,"num_object_clones":270,"num_object_copies":1632,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":816,"num_whiteouts":247,"num_read":78416,"num_read_kb":244670,"num_write":8717,"num_write_kb":140027,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":18486782,"num_keys_recovered":80,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6631,"ondisk_log_size":6631,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2463796,"kb_used_data":35680,"kb_used_omap":13,"kb_used_meta":2427890,"kb_avail":752510924,"statfs":{"total":773094113280,"available":770571186176,"internally_reserved":0,"allocated":36536320,"data_stored":276930900,"data_compressed":6302337,"data_compressed_allocated":17985536,"data_compressed_original":263246956,"omap_allocated":13632,"internal_metadata":2486160064},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":136496788,"num_objects":149,"num_object_clones":48,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":40,"num_read":15078,"num_read_kb":44112,"num_write":1611,"num_write_kb":27211,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023593"},"pg_stats":[{"pgid":"4.1f","version":"60'155","reported_seq":1651,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.902994+0000","last_change":"2024-04-24T18:07:38.288543+0000","last_active":"2024-04-24T18:08:11.902994+0000","last_peered":"2024-04-24T18:08:11.902994+0000","last_clean":"2024-04-24T18:08:11.902994+0000","last_became_active":"2024-04-24T18:07:36.550017+0000","last_became_peered":"2024-04-24T18:07:36.550017+0000","last_unstale":"2024-04-24T18:08:11.902994+0000","last_undegraded":"2024-04-24T18:08:11.902994+0000","last_fullsized":"2024-04-24T18:08:11.902994+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":155,"log_dups_size":0,"ondisk_log_size":155,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:32:03.728352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":24272994,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":6,"num_read":2044,"num_read_kb":11136,"num_write":208,"num_write_kb":3715,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"60'653","reported_seq":1697,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.899969+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:08:11.899969+0000","last_peered":"2024-04-24T18:08:11.899969+0000","last_clean":"2024-04-24T18:08:11.899969+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:08:11.899969+0000","last_undegraded":"2024-04-24T18:08:11.899969+0000","last_fullsized":"2024-04-24T18:08:11.899969+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":17882416,"num_objects":28,"num_object_clones":10,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":8,"num_read":4359,"num_read_kb":6852,"num_write":1131,"num_write_kb":5582,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"60'83","reported_seq":282,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.098232+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:08:12.098232+0000","last_peered":"2024-04-24T18:08:12.098232+0000","last_clean":"2024-04-24T18:08:12.098232+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:08:12.098232+0000","last_undegraded":"2024-04-24T18:08:12.098232+0000","last_fullsized":"2024-04-24T18:08:12.098232+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":6422982,"num_objects":19,"num_object_clones":7,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":154,"num_read_kb":3816,"num_write":104,"num_write_kb":3770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"60'306","reported_seq":3525,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.267848+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:08:12.267848+0000","last_peered":"2024-04-24T18:08:12.267848+0000","last_clean":"2024-04-24T18:08:12.267848+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:08:12.267848+0000","last_undegraded":"2024-04-24T18:08:12.267848+0000","last_fullsized":"2024-04-24T18:08:12.267848+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":21248717,"num_objects":29,"num_object_clones":11,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":8,"num_read":4194,"num_read_kb":7481,"num_write":447,"num_write_kb":5566,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"60'148","reported_seq":2954,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.098656+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:08:12.098656+0000","last_peered":"2024-04-24T18:08:12.098656+0000","last_clean":"2024-04-24T18:08:12.098656+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:08:12.098656+0000","last_undegraded":"2024-04-24T18:08:12.098656+0000","last_fullsized":"2024-04-24T18:08:12.098656+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":32534140,"num_objects":35,"num_object_clones":14,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":6,"num_read":2837,"num_read_kb":19154,"num_write":185,"num_write_kb":11787,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"60'156","reported_seq":4316,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.352281+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:08:12.352281+0000","last_peered":"2024-04-24T18:08:12.352281+0000","last_clean":"2024-04-24T18:08:12.352281+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:08:12.352281+0000","last_undegraded":"2024-04-24T18:08:12.352281+0000","last_fullsized":"2024-04-24T18:08:12.352281+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":15784183,"num_objects":36,"num_object_clones":12,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":14,"num_read":4195,"num_read_kb":11552,"num_write":158,"num_write_kb":7232,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"60'181","reported_seq":3563,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.330705+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:08:12.330705+0000","last_peered":"2024-04-24T18:08:12.330705+0000","last_clean":"2024-04-24T18:08:12.330705+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:08:12.330705+0000","last_undegraded":"2024-04-24T18:08:12.330705+0000","last_fullsized":"2024-04-24T18:08:12.330705+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":13595601,"num_objects":19,"num_object_clones":4,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":5,"num_read":4108,"num_read_kb":12022,"num_write":192,"num_write_kb":2678,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"60'129","reported_seq":3836,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.192105+0000","last_change":"2024-04-24T18:07:38.291282+0000","last_active":"2024-04-24T18:08:12.192105+0000","last_peered":"2024-04-24T18:08:12.192105+0000","last_clean":"2024-04-24T18:08:12.192105+0000","last_became_active":"2024-04-24T18:07:36.548455+0000","last_became_peered":"2024-04-24T18:07:36.548455+0000","last_unstale":"2024-04-24T18:08:12.192105+0000","last_undegraded":"2024-04-24T18:08:12.192105+0000","last_fullsized":"2024-04-24T18:08:12.192105+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":129,"log_dups_size":0,"ondisk_log_size":129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:07:50.006764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":28341504,"num_objects":31,"num_object_clones":11,"num_object_copies":62,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":9,"num_read":3722,"num_read_kb":14520,"num_write":154,"num_write_kb":8652,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"60'145","reported_seq":722,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.022776+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:08:12.022776+0000","last_peered":"2024-04-24T18:08:12.022776+0000","last_clean":"2024-04-24T18:08:12.022776+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:08:12.022776+0000","last_undegraded":"2024-04-24T18:08:12.022776+0000","last_fullsized":"2024-04-24T18:08:12.022776+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":145,"log_dups_size":0,"ondisk_log_size":145,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":21696936,"num_objects":25,"num_object_clones":11,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":7,"num_read":597,"num_read_kb":5928,"num_write":169,"num_write_kb":5994,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"60'320","reported_seq":362,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.371465+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:08:12.371465+0000","last_peered":"2024-04-24T18:08:12.371465+0000","last_clean":"2024-04-24T18:08:12.371465+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:08:12.371465+0000","last_undegraded":"2024-04-24T18:08:12.371465+0000","last_fullsized":"2024-04-24T18:08:12.371465+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":320,"log_dups_size":0,"ondisk_log_size":320,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":14106787,"num_objects":24,"num_object_clones":7,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":8,"num_read":1503,"num_read_kb":2470,"num_write":531,"num_write_kb":3210,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.742376+0000","last_change":"2024-04-24T18:07:20.402407+0000","last_active":"2024-04-24T18:08:10.742376+0000","last_peered":"2024-04-24T18:08:10.742376+0000","last_clean":"2024-04-24T18:08:10.742376+0000","last_became_active":"2024-04-24T18:07:20.402033+0000","last_became_peered":"2024-04-24T18:07:20.402033+0000","last_unstale":"2024-04-24T18:08:10.742376+0000","last_undegraded":"2024-04-24T18:08:10.742376+0000","last_fullsized":"2024-04-24T18:08:10.742376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:04:30.141825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.742378+0000","last_change":"2024-04-24T18:07:20.403096+0000","last_active":"2024-04-24T18:08:10.742378+0000","last_peered":"2024-04-24T18:08:10.742378+0000","last_clean":"2024-04-24T18:08:10.742378+0000","last_became_active":"2024-04-24T18:07:20.402839+0000","last_became_peered":"2024-04-24T18:07:20.402839+0000","last_unstale":"2024-04-24T18:08:10.742378+0000","last_undegraded":"2024-04-24T18:08:10.742378+0000","last_fullsized":"2024-04-24T18:08:10.742378+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:46:38.936107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"60'187","reported_seq":2665,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.358477+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:08:12.358477+0000","last_peered":"2024-04-24T18:08:12.358477+0000","last_clean":"2024-04-24T18:08:12.358477+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:08:12.358477+0000","last_undegraded":"2024-04-24T18:08:12.358477+0000","last_fullsized":"2024-04-24T18:08:12.358477+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":187,"log_dups_size":0,"ondisk_log_size":187,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":34637299,"num_objects":35,"num_object_clones":11,"num_object_copies":70,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":35,"num_whiteouts":12,"num_read":2757,"num_read_kb":8030,"num_write":217,"num_write_kb":4425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.679493+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:10.679493+0000","last_peered":"2024-04-24T18:08:10.679493+0000","last_clean":"2024-04-24T18:08:10.679493+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:10.679493+0000","last_undegraded":"2024-04-24T18:08:10.679493+0000","last_fullsized":"2024-04-24T18:08:10.679493+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":52,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.687899+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:10.687899+0000","last_peered":"2024-04-24T18:08:10.687899+0000","last_clean":"2024-04-24T18:08:10.687899+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:10.687899+0000","last_undegraded":"2024-04-24T18:08:10.687899+0000","last_fullsized":"2024-04-24T18:08:10.687899+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"60'156","reported_seq":382,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.900089+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:08:11.900089+0000","last_peered":"2024-04-24T18:08:11.900089+0000","last_clean":"2024-04-24T18:08:11.900089+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:08:11.900089+0000","last_undegraded":"2024-04-24T18:08:11.900089+0000","last_fullsized":"2024-04-24T18:08:11.900089+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":156,"log_dups_size":0,"ondisk_log_size":156,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":3622601,"num_objects":18,"num_object_clones":6,"num_object_copies":36,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":18,"num_whiteouts":7,"num_read":828,"num_read_kb":903,"num_write":174,"num_write_kb":806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.682967+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:10.682967+0000","last_peered":"2024-04-24T18:08:10.682967+0000","last_clean":"2024-04-24T18:08:10.682967+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:10.682967+0000","last_undegraded":"2024-04-24T18:08:10.682967+0000","last_fullsized":"2024-04-24T18:08:10.682967+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:13:26.133189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.772020+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:10.772020+0000","last_peered":"2024-04-24T18:08:10.772020+0000","last_clean":"2024-04-24T18:08:10.772020+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:10.772020+0000","last_undegraded":"2024-04-24T18:08:10.772020+0000","last_fullsized":"2024-04-24T18:08:10.772020+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"60'441","reported_seq":3916,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.901957+0000","last_change":"2024-04-24T18:07:38.288298+0000","last_active":"2024-04-24T18:08:11.901957+0000","last_peered":"2024-04-24T18:08:11.901957+0000","last_clean":"2024-04-24T18:08:11.901957+0000","last_became_active":"2024-04-24T18:07:36.549561+0000","last_became_peered":"2024-04-24T18:07:36.549561+0000","last_unstale":"2024-04-24T18:08:11.901957+0000","last_undegraded":"2024-04-24T18:08:11.901957+0000","last_fullsized":"2024-04-24T18:08:11.901957+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":441,"log_dups_size":0,"ondisk_log_size":441,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:05:48.800593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":17685506,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":6,"num_read":6343,"num_read_kb":16293,"num_write":603,"num_write_kb":4922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.677346+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:10.677346+0000","last_peered":"2024-04-24T18:08:10.677346+0000","last_clean":"2024-04-24T18:08:10.677346+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:10.677346+0000","last_undegraded":"2024-04-24T18:08:10.677346+0000","last_fullsized":"2024-04-24T18:08:10.677346+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":51,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.771785+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:10.771785+0000","last_peered":"2024-04-24T18:08:10.771785+0000","last_clean":"2024-04-24T18:08:10.771785+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:10.771785+0000","last_undegraded":"2024-04-24T18:08:10.771785+0000","last_fullsized":"2024-04-24T18:08:10.771785+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"59'131","reported_seq":411,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.842369+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:08:11.842369+0000","last_peered":"2024-04-24T18:08:11.842369+0000","last_clean":"2024-04-24T18:08:11.842369+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:08:11.842369+0000","last_undegraded":"2024-04-24T18:08:11.842369+0000","last_fullsized":"2024-04-24T18:08:11.842369+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":131,"log_dups_size":0,"ondisk_log_size":131,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":7706005,"num_objects":24,"num_object_clones":9,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":7,"num_read":1799,"num_read_kb":6102,"num_write":284,"num_write_kb":2979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":49,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.677536+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:10.677536+0000","last_peered":"2024-04-24T18:08:10.677536+0000","last_clean":"2024-04-24T18:08:10.677536+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:10.677536+0000","last_undegraded":"2024-04-24T18:08:10.677536+0000","last_fullsized":"2024-04-24T18:08:10.677536+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:43:25.729450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":249,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.772030+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:10.772030+0000","last_peered":"2024-04-24T18:08:10.772030+0000","last_clean":"2024-04-24T18:08:10.772030+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:10.772030+0000","last_undegraded":"2024-04-24T18:08:10.772030+0000","last_fullsized":"2024-04-24T18:08:10.772030+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":23,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.677448+0000","last_change":"2024-04-24T18:07:56.624753+0000","last_active":"2024-04-24T18:08:10.677448+0000","last_peered":"2024-04-24T18:08:10.677448+0000","last_clean":"2024-04-24T18:08:10.677448+0000","last_became_active":"2024-04-24T18:07:56.624464+0000","last_became_peered":"2024-04-24T18:07:56.624464+0000","last_unstale":"2024-04-24T18:08:10.677448+0000","last_undegraded":"2024-04-24T18:08:10.677448+0000","last_fullsized":"2024-04-24T18:08:10.677448+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:06:20.390454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"60'209","reported_seq":4758,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.098736+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:08:12.098736+0000","last_peered":"2024-04-24T18:08:12.098736+0000","last_clean":"2024-04-24T18:08:12.098736+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:08:12.098736+0000","last_undegraded":"2024-04-24T18:08:12.098736+0000","last_fullsized":"2024-04-24T18:08:12.098736+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":209,"log_dups_size":0,"ondisk_log_size":209,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":19002662,"num_objects":25,"num_object_clones":7,"num_object_copies":50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":25,"num_whiteouts":6,"num_read":5801,"num_read_kb":10797,"num_write":215,"num_write_kb":3577,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"59'87","reported_seq":307,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.310132+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:08:11.310132+0000","last_peered":"2024-04-24T18:08:11.310132+0000","last_clean":"2024-04-24T18:08:11.310132+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:08:11.310132+0000","last_undegraded":"2024-04-24T18:08:11.310132+0000","last_fullsized":"2024-04-24T18:08:11.310132+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":10020979,"num_objects":15,"num_object_clones":4,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":7,"num_read":248,"num_read_kb":643,"num_write":85,"num_write_kb":1522,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":52,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.683172+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:10.683172+0000","last_peered":"2024-04-24T18:08:10.683172+0000","last_clean":"2024-04-24T18:08:10.683172+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:10.683172+0000","last_undegraded":"2024-04-24T18:08:10.683172+0000","last_fullsized":"2024-04-24T18:08:10.683172+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"60'138","reported_seq":1552,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.726933+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:08:11.726933+0000","last_peered":"2024-04-24T18:08:11.726933+0000","last_clean":"2024-04-24T18:08:11.726933+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:08:11.726933+0000","last_undegraded":"2024-04-24T18:08:11.726933+0000","last_fullsized":"2024-04-24T18:08:11.726933+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":20016101,"num_objects":29,"num_object_clones":9,"num_object_copies":58,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":10,"num_read":1481,"num_read_kb":4572,"num_write":143,"num_write_kb":1191,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":52,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:10.679640+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:10.679640+0000","last_peered":"2024-04-24T18:08:10.679640+0000","last_clean":"2024-04-24T18:08:10.679640+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:10.679640+0000","last_undegraded":"2024-04-24T18:08:10.679640+0000","last_fullsized":"2024-04-24T18:08:10.679640+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"60'133","reported_seq":3090,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.262385+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:08:12.262385+0000","last_peered":"2024-04-24T18:08:12.262385+0000","last_clean":"2024-04-24T18:08:12.262385+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:08:12.262385+0000","last_undegraded":"2024-04-24T18:08:12.262385+0000","last_fullsized":"2024-04-24T18:08:12.262385+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":133,"log_dups_size":0,"ondisk_log_size":133,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":14396879,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":12,"num_read":3460,"num_read_kb":4765,"num_write":121,"num_write_kb":2561,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"60'138","reported_seq":1799,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.875104+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:08:11.875104+0000","last_peered":"2024-04-24T18:08:11.875104+0000","last_clean":"2024-04-24T18:08:11.875104+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:08:11.875104+0000","last_undegraded":"2024-04-24T18:08:11.875104+0000","last_fullsized":"2024-04-24T18:08:11.875104+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":138,"log_dups_size":0,"ondisk_log_size":138,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":30511871,"num_objects":26,"num_object_clones":8,"num_object_copies":52,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":7,"num_read":1814,"num_read_kb":7162,"num_write":182,"num_write_kb":4115,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"60'278","reported_seq":747,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.907029+0000","last_change":"2024-04-24T18:07:56.696872+0000","last_active":"2024-04-24T18:08:11.907029+0000","last_peered":"2024-04-24T18:08:11.907029+0000","last_clean":"2024-04-24T18:08:11.907029+0000","last_became_active":"2024-04-24T18:07:56.627442+0000","last_became_peered":"2024-04-24T18:07:56.627442+0000","last_unstale":"2024-04-24T18:08:11.907029+0000","last_undegraded":"2024-04-24T18:08:11.907029+0000","last_fullsized":"2024-04-24T18:08:11.907029+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":278,"log_dups_size":0,"ondisk_log_size":278,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:35.221342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":10478912,"num_objects":14,"num_object_clones":3,"num_object_copies":28,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":14,"num_whiteouts":3,"num_read":1294,"num_read_kb":1330,"num_write":403,"num_write_kb":2498,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1623115,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"60'91","reported_seq":917,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.897768+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:08:11.897768+0000","last_peered":"2024-04-24T18:08:11.897768+0000","last_clean":"2024-04-24T18:08:11.897768+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:08:11.897768+0000","last_undegraded":"2024-04-24T18:08:11.897768+0000","last_fullsized":"2024-04-24T18:08:11.897768+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":91,"log_dups_size":0,"ondisk_log_size":91,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":9814362,"num_objects":17,"num_object_clones":4,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":5,"num_read":769,"num_read_kb":1536,"num_write":90,"num_write_kb":2357,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"60'134","reported_seq":572,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.024177+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:08:12.024177+0000","last_peered":"2024-04-24T18:08:12.024177+0000","last_clean":"2024-04-24T18:08:12.024177+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:08:12.024177+0000","last_undegraded":"2024-04-24T18:08:12.024177+0000","last_fullsized":"2024-04-24T18:08:12.024177+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":14774119,"num_objects":19,"num_object_clones":6,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":6,"num_read":678,"num_read_kb":3431,"num_write":173,"num_write_kb":5424,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"60'134","reported_seq":698,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.989713+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:08:11.989713+0000","last_peered":"2024-04-24T18:08:11.989713+0000","last_clean":"2024-04-24T18:08:11.989713+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:08:11.989713+0000","last_undegraded":"2024-04-24T18:08:11.989713+0000","last_fullsized":"2024-04-24T18:08:11.989713+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":134,"log_dups_size":0,"ondisk_log_size":134,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":23278994,"num_objects":36,"num_object_clones":13,"num_object_copies":72,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":36,"num_whiteouts":12,"num_read":835,"num_read_kb":1995,"num_write":116,"num_write_kb":1358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"60'181","reported_seq":1107,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.904597+0000","last_change":"2024-04-24T18:07:38.291549+0000","last_active":"2024-04-24T18:08:11.904597+0000","last_peered":"2024-04-24T18:08:11.904597+0000","last_clean":"2024-04-24T18:08:11.904597+0000","last_became_active":"2024-04-24T18:07:36.549997+0000","last_became_peered":"2024-04-24T18:07:36.549997+0000","last_unstale":"2024-04-24T18:08:11.904597+0000","last_undegraded":"2024-04-24T18:08:11.904597+0000","last_fullsized":"2024-04-24T18:08:11.904597+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":181,"log_dups_size":0,"ondisk_log_size":181,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:05:15.050671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":7170363,"num_objects":27,"num_object_clones":11,"num_object_copies":54,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":27,"num_whiteouts":8,"num_read":1610,"num_read_kb":3597,"num_write":186,"num_write_kb":2745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"60'367","reported_seq":1796,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.282928+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:08:12.282928+0000","last_peered":"2024-04-24T18:08:12.282928+0000","last_clean":"2024-04-24T18:08:12.282928+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:08:12.282928+0000","last_undegraded":"2024-04-24T18:08:12.282928+0000","last_fullsized":"2024-04-24T18:08:12.282928+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":367,"log_dups_size":0,"ondisk_log_size":367,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":11111637,"num_objects":24,"num_object_clones":6,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":5,"num_read":3627,"num_read_kb":12980,"num_write":514,"num_write_kb":4852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"60'175","reported_seq":1399,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.307767+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:08:12.307767+0000","last_peered":"2024-04-24T18:08:12.307767+0000","last_clean":"2024-04-24T18:08:12.307767+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:08:12.307767+0000","last_undegraded":"2024-04-24T18:08:12.307767+0000","last_fullsized":"2024-04-24T18:08:12.307767+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":175,"log_dups_size":0,"ondisk_log_size":175,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":10865259,"num_objects":34,"num_object_clones":15,"num_object_copies":68,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":34,"num_whiteouts":14,"num_read":1824,"num_read_kb":3656,"num_write":166,"num_write_kb":3047,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"60'186","reported_seq":2510,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.334565+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:08:12.334565+0000","last_peered":"2024-04-24T18:08:12.334565+0000","last_clean":"2024-04-24T18:08:12.334565+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:08:12.334565+0000","last_undegraded":"2024-04-24T18:08:12.334565+0000","last_fullsized":"2024-04-24T18:08:12.334565+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":186,"log_dups_size":0,"ondisk_log_size":186,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":60915100,"num_objects":32,"num_object_clones":12,"num_object_copies":64,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":32,"num_whiteouts":9,"num_read":2940,"num_read_kb":27104,"num_write":238,"num_write_kb":15543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"60'153","reported_seq":495,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.367694+0000","last_change":"2024-04-24T18:07:56.675621+0000","last_active":"2024-04-24T18:08:12.367694+0000","last_peered":"2024-04-24T18:08:12.367694+0000","last_clean":"2024-04-24T18:08:12.367694+0000","last_became_active":"2024-04-24T18:07:56.624554+0000","last_became_peered":"2024-04-24T18:07:56.624554+0000","last_unstale":"2024-04-24T18:08:12.367694+0000","last_undegraded":"2024-04-24T18:08:12.367694+0000","last_fullsized":"2024-04-24T18:08:12.367694+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":153,"log_dups_size":0,"ondisk_log_size":153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:22:23.574241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":11425229,"num_objects":21,"num_object_clones":5,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":4,"num_read":2730,"num_read_kb":10870,"num_write":160,"num_write_kb":4156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2974171,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"59'106","reported_seq":2782,"reported_epoch":59,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.310192+0000","last_change":"2024-04-24T18:07:38.292302+0000","last_active":"2024-04-24T18:08:11.310192+0000","last_peered":"2024-04-24T18:08:11.310192+0000","last_clean":"2024-04-24T18:08:11.310192+0000","last_became_active":"2024-04-24T18:07:36.549147+0000","last_became_peered":"2024-04-24T18:07:36.549147+0000","last_unstale":"2024-04-24T18:08:11.310192+0000","last_undegraded":"2024-04-24T18:08:11.310192+0000","last_fullsized":"2024-04-24T18:08:11.310192+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":106,"log_dups_size":0,"ondisk_log_size":106,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:02.455780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":6518858,"num_objects":20,"num_object_clones":7,"num_object_copies":40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":20,"num_whiteouts":7,"num_read":2670,"num_read_kb":5251,"num_write":103,"num_write_kb":3637,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"60'176","reported_seq":1496,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.304814+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:08:12.304814+0000","last_peered":"2024-04-24T18:08:12.304814+0000","last_clean":"2024-04-24T18:08:12.304814+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:08:12.304814+0000","last_undegraded":"2024-04-24T18:08:12.304814+0000","last_fullsized":"2024-04-24T18:08:12.304814+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":176,"log_dups_size":0,"ondisk_log_size":176,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":22759733,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":6,"num_read":2404,"num_read_kb":6278,"num_write":173,"num_write_kb":3092,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"60'102","reported_seq":1590,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:12.074220+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:08:12.074220+0000","last_peered":"2024-04-24T18:08:12.074220+0000","last_clean":"2024-04-24T18:08:12.074220+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:08:12.074220+0000","last_undegraded":"2024-04-24T18:08:12.074220+0000","last_fullsized":"2024-04-24T18:08:12.074220+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":102,"log_dups_size":0,"ondisk_log_size":102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":4520721,"num_objects":22,"num_object_clones":8,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":10,"num_read":1483,"num_read_kb":4350,"num_write":99,"num_write_kb":2389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"60'535","reported_seq":1529,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-04-24T18:08:11.906520+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:08:11.906520+0000","last_peered":"2024-04-24T18:08:11.906520+0000","last_clean":"2024-04-24T18:08:11.906520+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:08:11.906520+0000","last_undegraded":"2024-04-24T18:08:11.906520+0000","last_fullsized":"2024-04-24T18:08:11.906520+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":25609763,"num_objects":28,"num_object_clones":8,"num_object_copies":56,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":28,"num_whiteouts":7,"num_read":3192,"num_read_kb":7994,"num_write":834,"num_write_kb":2229,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":572729213,"num_objects":814,"num_object_clones":270,"num_object_copies":1628,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":814,"num_whiteouts":247,"num_read":78300,"num_read_kb":244570,"num_write":8558,"num_write_kb":137611,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":17306046,"num_keys_recovered":80,"num_objects_omap":23,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":30941184,"data_stored":274932000,"data_compressed":6301427,"data_compressed_allocated":18063360,"data_compressed_original":263935084,"omap_allocated":0,"internal_metadata":0},"log_size":6514,"ondisk_log_size":6514,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574862,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305424,"kb_used_data":3968,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94066416,"statfs":{"total":96636764160,"available":96324009984,"internally_reserved":0,"allocated":4063232,"data_stored":37398578,"data_compressed":556273,"data_compressed_allocated":2166784,"data_compressed_original":36131589,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28699999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32700000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305156,"kb_used_data":3708,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94066684,"statfs":{"total":96636764160,"available":96324284416,"internally_reserved":0,"allocated":3796992,"data_stored":28361506,"data_compressed":776140,"data_compressed_allocated":2023424,"data_compressed_original":27074229,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34200000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.499}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44800000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58699999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77400000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574861,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312032,"kb_used_data":6488,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94059808,"statfs":{"total":96636764160,"available":96317243392,"internally_reserved":0,"allocated":6643712,"data_stored":40217421,"data_compressed":903457,"data_compressed_allocated":2744320,"data_compressed_original":37391896,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.436}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32000000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56399999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49099999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574862,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308076,"kb_used_data":2520,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063764,"statfs":{"total":96636764160,"available":96321294336,"internally_reserved":0,"allocated":2580480,"data_stored":17338668,"data_compressed":365305,"data_compressed_allocated":1060864,"data_compressed_original":16300816,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30299999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42199999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39300000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41099999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55500000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.378}]}]},{"osd":2,"up_from":13,"seq":55834574861,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303232,"kb_used_data":1780,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068608,"statfs":{"total":96636764160,"available":96326254592,"internally_reserved":0,"allocated":1822720,"data_stored":8100723,"data_compressed":248060,"data_compressed_allocated":651264,"data_compressed_original":7393280,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.502}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.26300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55800000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47399999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574862,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313156,"kb_used_data":7548,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94058684,"statfs":{"total":96636764160,"available":96316092416,"internally_reserved":0,"allocated":7729152,"data_stored":73541936,"data_compressed":1451418,"data_compressed_allocated":4452352,"data_compressed_original":70828157,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91300000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34699999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.746}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79300000000000004}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83999999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574862,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311532,"kb_used_data":5928,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94060308,"statfs":{"total":96636764160,"available":96317755392,"internally_reserved":0,"allocated":6070272,"data_stored":52729575,"data_compressed":1577610,"data_compressed_allocated":3567616,"data_compressed_original":50750275,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38100000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35599999999999998}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27400000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.37}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.439}]}]},{"osd":5,"up_from":13,"seq":55834574862,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305188,"kb_used_data":3740,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94066652,"statfs":{"total":96636764160,"available":96324251648,"internally_reserved":0,"allocated":3829760,"data_stored":19242493,"data_compressed":424074,"data_compressed_allocated":1318912,"data_compressed_original":17376714,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55400000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38300000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53400000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40699999999999997}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.5}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56699999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1949696,"data_stored":17329292,"data_compressed":368327,"data_compressed_allocated":1097728,"data_compressed_original":16603920,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":5775360,"data_stored":39896630,"data_compressed":900517,"data_compressed_allocated":2744320,"data_compressed_original":37506584,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":1155072,"data_stored":8042106,"data_compressed":250471,"data_compressed_allocated":667648,"data_compressed_original":7655424,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":7086080,"data_stored":73520156,"data_compressed":1453948,"data_compressed_allocated":4476928,"data_compressed_original":71106685,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":5386240,"data_stored":52412721,"data_compressed":1577677,"data_compressed_allocated":3571712,"data_compressed_original":50758467,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":3137536,"data_stored":18936850,"data_compressed":424275,"data_compressed_allocated":1331200,"data_compressed_original":17401290,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3141632,"data_stored":28312954,"data_compressed":778603,"data_compressed_allocated":2043904,"data_compressed_original":27344565,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":3309568,"data_stored":36481291,"data_compressed":547609,"data_compressed_allocated":2129920,"data_compressed_original":35558149,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:08:14.539 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-24T18:08:14.539 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:08:14.539 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-24T18:08:14.539 INFO:tasks.thrashosds.thrasher:Setting osd 7 primary_affinity to 1.000000 2024-04-24T18:08:14.539 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-24T18:08:14.543 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:08:14.584 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:14.592+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:14.659 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:14.667+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:14.709 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:14.709 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:14.720 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:08:14.750 INFO:teuthology.orchestra.run.smithi049.stdout:leaving image image_client.0-clone21 intact 2024-04-24T18:08:14.761 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:14.769+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:14.861 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:14.869+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:14.874 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #20, image image_client.0-clone20 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent21 2024-04-24T18:08:14.885 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:14.885 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:14.895 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:14.899 INFO:teuthology.orchestra.run.smithi049.stdout:2762 punch from 0x9b70779 to 0x9b7d439, (0xccc0 bytes) 2024-04-24T18:08:14.902 INFO:teuthology.orchestra.run.smithi049.stdout:2763 read 0x17b4df0 thru 0x17b934a (0x455b bytes) 2024-04-24T18:08:14.902 INFO:teuthology.orchestra.run.smithi049.stdout:2764 write 0x6c1b88a thru 0x6c26320 (0xaa97 bytes) 2024-04-24T18:08:14.906 INFO:teuthology.orchestra.run.smithi049.stdout:2765 write 0x5702f7d thru 0x57097b1 (0x6835 bytes) 2024-04-24T18:08:14.909 INFO:teuthology.orchestra.run.smithi049.stdout:2766 punch from 0x990da92 to 0x99147e6, (0x6d54 bytes) 2024-04-24T18:08:14.909 INFO:teuthology.orchestra.run.smithi049.stdout:2768 writesame 0x5d7c7 thru 0x6bd76 data_size 0xd2(0xe5b0 bytes) 2024-04-24T18:08:14.962 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:14.970+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:14.981 INFO:teuthology.orchestra.run.smithi049.stdout:2769 write 0xc033140 thru 0xc04172a (0xe5eb bytes) 2024-04-24T18:08:14.997 INFO:teuthology.orchestra.run.smithi049.stdout:2770 trunc from 0xc04172b to 0xdbae9f 2024-04-24T18:08:15.030 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:15.030 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:15.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:15.059 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:08:15.060 INFO:teuthology.orchestra.run.smithi049.stdout:2771 read 0xc9bdda thru 0xca2309 (0x6530 bytes) 2024-04-24T18:08:15.060 INFO:teuthology.orchestra.run.smithi049.stdout:2772 read 0xd795a0 thru 0xd824f6 (0x8f57 bytes) 2024-04-24T18:08:15.078 INFO:teuthology.orchestra.run.smithi049.stdout:2773 write 0xd280888 thru 0xd287bb3 (0x732c bytes) 2024-04-24T18:08:15.079 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:15.087+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:15.081 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:15.081 INFO:teuthology.orchestra.run.smithi049.stdout:2776 read 0x4ffd752 thru 0x500d62b (0xfeda bytes) 2024-04-24T18:08:15.081 INFO:teuthology.orchestra.run.smithi049.stdout:2777 read 0x61faa09 thru 0x6201e61 (0x7459 bytes) 2024-04-24T18:08:15.090 INFO:teuthology.orchestra.run.smithi049.stdout:2778 write 0xe3c95e4 thru 0xe3d145f (0x7e7c bytes) 2024-04-24T18:08:15.093 INFO:teuthology.orchestra.run.smithi049.stdout:2779 punch from 0x518562b to 0x5189d38, (0x470d bytes) 2024-04-24T18:08:15.093 INFO:teuthology.orchestra.run.smithi049.stdout:2780 read 0xae9650f thru 0xae9d4c1 (0x6fb3 bytes) 2024-04-24T18:08:15.093 INFO:teuthology.orchestra.run.smithi049.stdout:2781 write 0xb9db9 thru 0xc148e (0x76d6 bytes) 2024-04-24T18:08:15.095 INFO:teuthology.orchestra.run.smithi049.stdout:2782 write 0x2a3e295 thru 0x2a4d67f (0xf3eb bytes) 2024-04-24T18:08:15.098 INFO:teuthology.orchestra.run.smithi049.stdout:2783 read 0x8b0bec9 thru 0x8b137ac (0x78e4 bytes) 2024-04-24T18:08:15.098 INFO:teuthology.orchestra.run.smithi049.stdout:2785 write 0x7bddeea thru 0x7be9da3 (0xbeba bytes) 2024-04-24T18:08:15.108 INFO:teuthology.orchestra.run.smithi049.stdout:2786 trunc from 0xe3d1460 to 0x829ed65 2024-04-24T18:08:15.119 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:15.119 INFO:teuthology.orchestra.run.smithi049.stdout:2788 write 0x266d129 thru 0x266d463 (0x33b bytes) 2024-04-24T18:08:15.122 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:15.122 INFO:teuthology.orchestra.run.smithi049.stdout:2790 write 0x75b5a8c thru 0x75bb7ba (0x5d2f bytes) 2024-04-24T18:08:15.126 INFO:teuthology.orchestra.run.smithi049.stdout:2791 read 0x1b94b88 thru 0x1b9c2d7 (0x7750 bytes) 2024-04-24T18:08:15.127 INFO:teuthology.orchestra.run.smithi049.stdout:2792 read 0x6fce8b thru 0x70c6a0 (0xf816 bytes) 2024-04-24T18:08:15.127 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:15.136 INFO:teuthology.orchestra.run.smithi049.stdout:2797 write 0xabdca02 thru 0xabe3265 (0x6864 bytes) 2024-04-24T18:08:15.139 INFO:teuthology.orchestra.run.smithi049.stdout:2800 writesame 0x1b1106b thru 0x1b1958e data_size 0x2(0x8524 bytes) 2024-04-24T18:08:15.158 INFO:teuthology.orchestra.run.smithi049.stdout:2801 trunc from 0xabe3266 to 0x2f81817 2024-04-24T18:08:15.163 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:15.171+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:15.185 INFO:teuthology.orchestra.run.smithi049.stdout:2802 write 0x91bfed4 thru 0x91c6767 (0x6894 bytes) 2024-04-24T18:08:15.186 INFO:teuthology.orchestra.run.smithi049.stdout:2803 read 0x18c7afa thru 0x18cc238 (0x473f bytes) 2024-04-24T18:08:15.195 INFO:teuthology.orchestra.run.smithi049.stdout:2804 write 0xba040aa thru 0xba0e8e5 (0xa83c bytes) 2024-04-24T18:08:15.200 INFO:teuthology.orchestra.run.smithi049.stdout:2806 read 0xab28ea0 thru 0xab2c2ed (0x344e bytes) 2024-04-24T18:08:15.200 INFO:teuthology.orchestra.run.smithi049.stdout:2807 punch from 0x4d931 to 0x53dd6, (0x64a5 bytes) 2024-04-24T18:08:15.200 INFO:teuthology.orchestra.run.smithi049.stdout:2808 read 0xc72b6d thru 0xc7f41c (0xc8b0 bytes) 2024-04-24T18:08:15.206 INFO:teuthology.orchestra.run.smithi049.stdout:2809 write 0xc941de7 thru 0xc9422c0 (0x4da bytes) 2024-04-24T18:08:15.207 INFO:teuthology.orchestra.run.smithi049.stdout:2810 clone 23 order 23 su 524288 sc 2 2024-04-24T18:08:15.214 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:15.214 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:15.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:15.266 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:15.274+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:15.366 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:15.374+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:15.388 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:15.388 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:15.398 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:15.411 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:15.466 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:15.475+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:15.533 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:15.533 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:15.544 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:15.567 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:15.575+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:15.590 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:15.608 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:15.627 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:15.646 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:15.665 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:15.668 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:15.676+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:15.674 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:15.674 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:15.675 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:15.675 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:15.675 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:15.683 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:15.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:15.706 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:08:15.707 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-24T18:08:15.708 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:15.737 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:15.738 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:15.768 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone22 from 0xc9422c1 (overlap 0xdbae9f) to 0x6dd74f 2024-04-24T18:08:15.769 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:15.777+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:15.842 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:15.842 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:15.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:15.842 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:15.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31810 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 2920448~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.157055388, 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052993206000000001, 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31810 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:15.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691740+0000", 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691739+0000", 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691742+0000", 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2800000000000002e-06 2024-04-24T18:08:15.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691747+0000", 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4319999999999998e-06 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.691750+0000", 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.247e-06 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744520+0000", 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052769498999999997 2024-04-24T18:08:15.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744623+0000", 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000102808 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.744733+0000", 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000110282 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.143470627999999, 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:08:15.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:15.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:08:15.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32297 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3233792~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:15.852 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.131096544999998, 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052981070999999998, 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32297 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717699+0000", 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717698+0000", 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717705+0000", 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5529999999999998e-06 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717711+0000", 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3849999999999999e-06 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717715+0000", 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9489999999999998e-06 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770506+0000", 2024-04-24T18:08:15.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052791267000000003 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770587+0000", 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0783000000000002e-05 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770680+0000", 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2497000000000006e-05 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32299 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3252224~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.131075542000001, 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053161606, 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32299 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.857 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717720+0000", 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717719+0000", 2024-04-24T18:08:15.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717722+0000", 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.43e-06 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717729+0000", 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.576e-06 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717733+0000", 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0119999999999999e-06 2024-04-24T18:08:15.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770707+0000", 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052974083999999998 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770791+0000", 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3418000000000007e-05 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770881+0000", 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0490999999999998e-05 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.860 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32301 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3260416~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.131057672000001, 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053336189999999999, 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32301 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717738+0000", 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717737+0000", 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717740+0000", 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4090000000000001e-06 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717746+0000", 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9989999999999997e-06 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717752+0000", 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0349999999999998e-06 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770915+0000", 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053162938 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.770989+0000", 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4567999999999995e-05 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771074+0000", 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4598000000000006e-05 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.864 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32303 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3270656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.130944593999999, 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053426976000000001, 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32303 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717851+0000", 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717850+0000", 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717854+0000", 2024-04-24T18:08:15.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0509999999999998e-06 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717863+0000", 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.346e-06 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717877+0000", 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3559e-05 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771102+0000", 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053224869000000001 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771184+0000", 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2564000000000004e-05 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771278+0000", 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3178999999999997e-05 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32305 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3278848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:15.868 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.130912425999998, 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053587411000000001, 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32305 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717883+0000", 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717882+0000", 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717885+0000", 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8679999999999999e-06 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717892+0000", 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5570000000000001e-06 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717896+0000", 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9360000000000001e-06 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771305+0000", 2024-04-24T18:08:15.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053409310000000002 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771373+0000", 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8417999999999994e-05 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771470+0000", 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6827e-05 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32307 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3289088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.130895384999999, 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053767057, 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32307 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.873 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717900+0000", 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717899+0000", 2024-04-24T18:08:15.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717902+0000", 2024-04-24T18:08:15.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5019999999999999e-06 2024-04-24T18:08:15.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717908+0000", 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7699999999999998e-06 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.717911+0000", 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4979999999999998e-06 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771498+0000", 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053586747999999997 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771571+0000", 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3168000000000001e-05 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.771667+0000", 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.5810000000000006e-05 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.877 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32311 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3307520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.129598999999999, 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053356705999999997, 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32311 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719196+0000", 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719199+0000", 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.88e-06 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719205+0000", 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719209+0000", 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0749999999999999e-06 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772008+0000", 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052798572000000002 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772048+0000", 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0238999999999999e-05 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772553+0000", 2024-04-24T18:08:15.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050474099999999996 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32313 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3315712~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.129580942, 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053105309000000003, 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32313 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719214+0000", 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719216+0000", 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4059999999999999e-06 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719223+0000", 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3010000000000003e-06 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719226+0000", 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7610000000000001e-06 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772130+0000", 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052903238999999998 2024-04-24T18:08:15.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772209+0000", 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9486000000000004e-05 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772320+0000", 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00011052099999999999 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32315 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3334144~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.129564298999998, 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053250991999999997, 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32315 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719231+0000", 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719233+0000", 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1789999999999998e-06 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719240+0000", 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5689999999999999e-06 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719244+0000", 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5870000000000001e-06 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772345+0000", 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053100972000000003 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772413+0000", 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8071000000000006e-05 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772482+0000", 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8950999999999998e-05 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32317 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3342336~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.129482238000001, 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053370316000000001, 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32317 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719313+0000", 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719312+0000", 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719316+0000", 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9400000000000004e-06 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719326+0000", 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2439999999999999e-06 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719331+0000", 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7049999999999996e-06 2024-04-24T18:08:15.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772583+0000", 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053251860999999998 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772628+0000", 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5289e-05 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772683+0000", 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4882000000000001e-05 2024-04-24T18:08:15.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32319 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3352576~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.129458894999999, 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053459060000000003, 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32319 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.896 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719336+0000", 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719338+0000", 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5490000000000001e-06 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719346+0000", 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5059999999999998e-06 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719350+0000", 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1999999999999996e-06 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772709+0000", 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053359182999999998 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772748+0000", 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8654e-05 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772795+0000", 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7437e-05 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32321 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3360768~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.129440684999999, 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053627222000000002, 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.900 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32321 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719355+0000", 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719354+0000", 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719361+0000", 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7449999999999998e-06 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719367+0000", 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3940000000000001e-06 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.719378+0000", 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1204e-05 2024-04-24T18:08:15.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772830+0000", 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053451920999999999 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772896+0000", 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5623000000000005e-05 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772982+0000", 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5729000000000005e-05 2024-04-24T18:08:15.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32327 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3389440~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.128424476999999, 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053038750000000003, 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32327 2024-04-24T18:08:15.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720371+0000", 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720370+0000", 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720373+0000", 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5830000000000001e-06 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720380+0000", 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1550000000000001e-06 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720384+0000", 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0260000000000001e-06 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773322+0000", 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052937495000000001 2024-04-24T18:08:15.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773364+0000", 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2561999999999997e-05 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773409+0000", 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5333000000000001e-05 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.909 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32329 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3397632~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.128406777999999, 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053169842000000002, 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32329 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:15.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720388+0000", 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720391+0000", 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7779999999999999e-06 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720396+0000", 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6729999999999997e-06 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720400+0000", 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7179999999999998e-06 2024-04-24T18:08:15.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773435+0000", 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053035169 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773497+0000", 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1715000000000001e-05 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773558+0000", 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1171000000000001e-05 2024-04-24T18:08:15.913 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32331 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3416064~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.128390649, 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053284320000000003, 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.914 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32331 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720405+0000", 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720404+0000", 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720406+0000", 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3190000000000002e-06 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720412+0000", 2024-04-24T18:08:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8359999999999997e-06 2024-04-24T18:08:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720416+0000", 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8500000000000004e-06 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773583+0000", 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053167076000000001 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773638+0000", 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5105999999999997e-05 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773689+0000", 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0521e-05 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32333 4.3 4:c63c7603:::rbd_data.11749ffb1524.0000000000000002:9 [read 3424256~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.919 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.128373036999999, 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053395479000000003, 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32333 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:15.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720422+0000", 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720424+0000", 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:08:15.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720430+0000", 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.720434+0000", 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8210000000000003e-06 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773721+0000", 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.053287297999999997 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773762+0000", 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1245000000000002e-05 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.773818+0000", 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5585999999999999e-05 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:32388 4.3 4:c61fab1f:::rbd_object_map.117432c990a0.000000000000000a:head [call rbd.object_map_update in=20b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 23.126270808000001, 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.055543581000000002, 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 32388 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722524+0000", 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:15.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722527+0000", 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0319999999999999e-06 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722535+0000", 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9170000000000006e-06 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.722539+0000", 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9709999999999998e-06 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774649+0000", 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.052110109000000002 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.774719+0000", 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9562999999999998e-05 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:15.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.775050+0000", 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000331078 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777149+0000", 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020996639999999998 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.777996+0000", 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00084650899999999996 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:15.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778027+0000", 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0818999999999997e-05 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.778068+0000", 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1338000000000003e-05 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 19.230995822000001, 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:15.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:08:15.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.935 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:15.936 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:15.936 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:15.937 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #21, image image_client.0-clone21 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent22 2024-04-24T18:08:15.938 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:15.938 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:15.946+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:16.038 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:16.047+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:16.119 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:16.119 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:16.119 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:16.119 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:16.130 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:16.171 INFO:teuthology.orchestra.run.smithi049.stdout:2811 writesame 0xa1d49fa thru 0xa1daf47 data_size 0x1(0x654e bytes) 2024-04-24T18:08:16.172 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:16.180+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:16.189 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.190 INFO:teuthology.orchestra.run.smithi049.stdout:2814 trunc from 0xc9422c1 to 0xbd6e838 2024-04-24T18:08:16.201 INFO:teuthology.orchestra.run.smithi049.stdout:2815 punch from 0xb317981 to 0xb32701e, (0xf69d bytes) 2024-04-24T18:08:16.201 INFO:teuthology.orchestra.run.smithi049.stdout:2816 write 0x6b79675 thru 0x6b7bc74 (0x2600 bytes) 2024-04-24T18:08:16.204 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.204 INFO:teuthology.orchestra.run.smithi049.stdout:2821 writesame 0xbffae8 thru 0xc0c469 data_size 0x1(0xc982 bytes) 2024-04-24T18:08:16.239 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:16.248+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:16.246 INFO:teuthology.orchestra.run.smithi049.stdout:2822 write 0xe890b41 thru 0xe89f693 (0xeb53 bytes) 2024-04-24T18:08:16.249 INFO:teuthology.orchestra.run.smithi049.stdout:2823 punch from 0xb39042 to 0xb42e23, (0x9de1 bytes) 2024-04-24T18:08:16.249 INFO:teuthology.orchestra.run.smithi049.stdout:2824 punch from 0x2b02b76 to 0x2b0c16b, (0x95f5 bytes) 2024-04-24T18:08:16.259 INFO:teuthology.orchestra.run.smithi049.stdout:2825 trunc from 0xe89f694 to 0x5980e3c 2024-04-24T18:08:16.272 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.284 INFO:teuthology.orchestra.run.smithi049.stdout:2830 write 0xb79d6f3 thru 0xb7a1088 (0x3996 bytes) 2024-04-24T18:08:16.287 INFO:teuthology.orchestra.run.smithi049.stdout:2831 punch from 0x696e273 to 0x697d0c7, (0xee54 bytes) 2024-04-24T18:08:16.290 INFO:teuthology.orchestra.run.smithi049.stdout:2832 trunc from 0xb7a1089 to 0x8a002e3 2024-04-24T18:08:16.299 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.299 INFO:teuthology.orchestra.run.smithi049.stdout:2834 read 0x4b1cb94 thru 0x4b2a4f1 (0xd95e bytes) 2024-04-24T18:08:16.299 INFO:teuthology.orchestra.run.smithi049.stdout:2835 writesame 0x19a7280 thru 0x19b42df data_size 0x10(0xd060 bytes) 2024-04-24T18:08:16.304 INFO:teuthology.orchestra.run.smithi049.stdout:2836 read 0xdf0a6d thru 0xdf390e (0x2ea2 bytes) 2024-04-24T18:08:16.316 INFO:teuthology.orchestra.run.smithi049.stdout:2837 write 0xedc4dd2 thru 0xedd4268 (0xf497 bytes) 2024-04-24T18:08:16.319 INFO:teuthology.orchestra.run.smithi049.stdout:2838 write 0x56e4817 thru 0x56edbac (0x9396 bytes) 2024-04-24T18:08:16.320 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:16.320 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:16.320 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:16.320 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:16.321 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:16.321 INFO:teuthology.orchestra.run.smithi049.stdout:2839 write 0xcf224de thru 0xcf2322d (0xd50 bytes) 2024-04-24T18:08:16.324 INFO:teuthology.orchestra.run.smithi049.stdout:2840 punch from 0xd2214ff to 0xd22f77f, (0xe280 bytes) 2024-04-24T18:08:16.324 INFO:teuthology.orchestra.run.smithi049.stdout:2841 write 0x8be2bb5 thru 0x8bf0f3a (0xe386 bytes) 2024-04-24T18:08:16.328 INFO:teuthology.orchestra.run.smithi049.stdout:2842 read 0xa88a69a thru 0xa891052 (0x69b9 bytes) 2024-04-24T18:08:16.328 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.328 INFO:teuthology.orchestra.run.smithi049.stdout:2844 write 0xec479ce thru 0xec5777f (0xfdb2 bytes) 2024-04-24T18:08:16.331 INFO:teuthology.orchestra.run.smithi049.stdout:2845 writesame 0x9bc2d93 thru 0x9bd2c09 data_size 0x1(0xfe77 bytes) 2024-04-24T18:08:16.333 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:16.340 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:16.348+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:16.365 INFO:teuthology.orchestra.run.smithi049.stdout:2846 punch from 0x33d5851 to 0x33e41a4, (0xe953 bytes) 2024-04-24T18:08:16.365 INFO:teuthology.orchestra.run.smithi049.stdout:2847 write 0xa3d522 thru 0xa4d519 (0xfff8 bytes) 2024-04-24T18:08:16.366 INFO:teuthology.orchestra.run.smithi049.stdout:2848 write 0x76ba5bf thru 0x76c87d6 (0xe218 bytes) 2024-04-24T18:08:16.370 INFO:teuthology.orchestra.run.smithi049.stdout:2851 read 0x419955b thru 0x41a4d0a (0xb7b0 bytes) 2024-04-24T18:08:16.370 INFO:teuthology.orchestra.run.smithi049.stdout:2853 writesame 0x220a82f thru 0x2210cc2 data_size 0x4(0x6494 bytes) 2024-04-24T18:08:16.377 INFO:teuthology.orchestra.run.smithi049.stdout:2854 write 0x20546ec thru 0x2062c67 (0xe57c bytes) 2024-04-24T18:08:16.378 INFO:teuthology.orchestra.run.smithi049.stdout:2855 read 0x79c2632 thru 0x79c6654 (0x4023 bytes) 2024-04-24T18:08:16.390 INFO:teuthology.orchestra.run.smithi049.stdout:2857 trunc from 0xedd4269 to 0x513bc07 2024-04-24T18:08:16.413 INFO:teuthology.orchestra.run.smithi049.stdout:2860 writesame 0xac61c37 thru 0xac64277 data_size 0x1(0x2641 bytes) 2024-04-24T18:08:16.413 INFO:teuthology.orchestra.run.smithi049.stdout:2862 read 0x9a8b331 thru 0x9a8f543 (0x4213 bytes) 2024-04-24T18:08:16.413 INFO:teuthology.orchestra.run.smithi049.stdout:2864 punch from 0xa65d573 to 0xa6659c5, (0x8452 bytes) 2024-04-24T18:08:16.422 INFO:teuthology.orchestra.run.smithi049.stdout:2865 trunc from 0xac64278 to 0x2ca8e7d 2024-04-24T18:08:16.441 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:16.450+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:16.450 INFO:teuthology.orchestra.run.smithi049.stdout:2869 write 0xce31fbc thru 0xce38c57 (0x6c9c bytes) 2024-04-24T18:08:16.453 INFO:teuthology.orchestra.run.smithi049.stdout:2870 trunc from 0xce38c58 to 0xd058dc3 2024-04-24T18:08:16.459 INFO:teuthology.orchestra.run.smithi049.stdout:2871 trunc from 0xd058dc3 to 0xeb7f0c5 2024-04-24T18:08:16.465 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.465 INFO:teuthology.orchestra.run.smithi049.stdout:2874 read 0xb074010 thru 0xb07ec0e (0xabff bytes) 2024-04-24T18:08:16.465 INFO:teuthology.orchestra.run.smithi049.stdout:2876 writesame 0x13e9af thru 0x145afb data_size 0x1(0x714d bytes) 2024-04-24T18:08:16.479 INFO:teuthology.orchestra.run.smithi049.stdout:2877 read 0x1f0d292 thru 0x1f148a3 (0x7612 bytes) 2024-04-24T18:08:16.479 INFO:teuthology.orchestra.run.smithi049.stdout:2878 read 0x33be4ae thru 0x33beb31 (0x684 bytes) 2024-04-24T18:08:16.494 INFO:teuthology.orchestra.run.smithi049.stdout:2879 trunc from 0xeb7f0c5 to 0xbfd106 2024-04-24T18:08:16.520 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:16.520 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:16.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:16.520 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:16.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40940 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819803452999999, 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036446457000000002, 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40940 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.521 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707548+0000", 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.251e-06 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707551+0000", 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0019999999999998e-06 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707554+0000", 2024-04-24T18:08:16.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743860+0000", 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036306756000000003 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743915+0000", 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4423999999999999e-05 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743994+0000", 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9207000000000002e-05 2024-04-24T18:08:16.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40941 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1490944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819769902000001, 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036634427999999997, 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40941 2024-04-24T18:08:16.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:16.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707583+0000", 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2089999999999999e-06 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707587+0000", 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1670000000000001e-06 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707590+0000", 2024-04-24T18:08:16.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.554e-06 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744033+0000", 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036443542000000002 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744112+0000", 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8578999999999994e-05 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744215+0000", 2024-04-24T18:08:16.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103607 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40942 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1523712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819758856, 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036756906999999998, 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.529 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40942 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707594+0000", 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7090000000000001e-06 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707596+0000", 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707598+0000", 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9149999999999999e-06 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744245+0000", 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036646570000000003 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744295+0000", 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0343000000000002e-05 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744349+0000", 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3661e-05 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40943 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819750177, 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036956462000000002, 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.533 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40943 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707600+0000", 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707602+0000", 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.215e-06 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707605+0000", 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0429999999999999e-06 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707607+0000", 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-06 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744380+0000", 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036773477999999998 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744464+0000", 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4048000000000006e-05 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744557+0000", 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3043999999999999e-05 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40944 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1589248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:16.537 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819742176999998, 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037054089999999998, 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40944 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707610+0000", 2024-04-24T18:08:16.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1319999999999999e-06 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707613+0000", 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707614+0000", 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8819999999999999e-06 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744569+0000", 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036954539000000002 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744613+0000", 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3671999999999998e-05 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744663+0000", 2024-04-24T18:08:16.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0145000000000002e-05 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40945 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819734276999998, 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037248315999999997, 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.542 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40945 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707616+0000", 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707620+0000", 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1999999999999999e-06 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707623+0000", 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1669999999999999e-06 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707625+0000", 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1110000000000002e-06 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744693+0000", 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037068564999999998 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744773+0000", 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9172999999999996e-05 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744865+0000", 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2305000000000005e-05 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40946 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819723734, 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037340299, 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40946 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707628+0000", 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2699999999999999e-06 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707631+0000", 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9519999999999999e-06 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707633+0000", 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8619999999999999e-06 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744876+0000", 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037242550999999999 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744918+0000", 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2546000000000002e-05 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744967+0000", 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9320999999999998e-05 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40947 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:16.550 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819715678000001, 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037537860999999999, 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40947 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707636+0000", 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.186e-06 2024-04-24T18:08:16.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707639+0000", 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707641+0000", 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.86e-06 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744997+0000", 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037356364000000003 2024-04-24T18:08:16.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745076+0000", 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8355000000000005e-05 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745173+0000", 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7379000000000004e-05 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40948 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1720320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819707754, 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037655182000000002, 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40948 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707644+0000", 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.269e-06 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707647+0000", 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9359999999999999e-06 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707649+0000", 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.815e-06 2024-04-24T18:08:16.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745200+0000", 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037551189999999998 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745246+0000", 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6214999999999997e-05 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745298+0000", 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1960999999999999e-05 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40949 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819699725, 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037846454000000002, 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40949 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707652+0000", 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.13e-06 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707655+0000", 2024-04-24T18:08:16.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707658+0000", 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5770000000000001e-06 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745329+0000", 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037671351999999998 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745408+0000", 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745498+0000", 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0043000000000005e-05 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40950 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:16.565 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819691044999999, 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037946947000000002, 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40950 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707661+0000", 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.15e-06 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707664+0000", 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.965e-06 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707666+0000", 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745508+0000", 2024-04-24T18:08:16.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037841977999999998 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745555+0000", 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7098999999999997e-05 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745607+0000", 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1591999999999997e-05 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40951 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819682565000001, 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038137931999999999, 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40951 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707670+0000", 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7010000000000001e-06 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707673+0000", 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9119999999999998e-06 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707675+0000", 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9429999999999999e-06 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745637+0000", 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037962379999999997 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745715+0000", 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7488000000000001e-05 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745806+0000", 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1723000000000003e-05 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.573 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40952 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819673939000001, 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038236762000000001, 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40952 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:16.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707678+0000", 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.093e-06 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707681+0000", 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707683+0000", 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.951e-06 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745817+0000", 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038133854000000002 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745862+0000", 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5027000000000001e-05 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745914+0000", 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2120999999999998e-05 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40953 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819665906000001, 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038424458000000002, 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40953 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:16.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707686+0000", 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.567e-06 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707689+0000", 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9220000000000001e-06 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707691+0000", 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9369999999999998e-06 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745943+0000", 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038252047999999997 2024-04-24T18:08:16.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746018+0000", 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4938000000000007e-05 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746109+0000", 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1255000000000006e-05 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.582 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40954 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1916928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819657397, 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038528139000000003, 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40954 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707693+0000", 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.065e-06 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707697+0000", 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.881e-06 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707699+0000", 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.88e-06 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746120+0000", 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038420937000000002 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746171+0000", 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0555e-05 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746222+0000", 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1025e-05 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40955 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819649577, 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718855000000003, 2024-04-24T18:08:16.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40955 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707702+0000", 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.212e-06 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707706+0000", 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.534e-06 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707708+0000", 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746254+0000", 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038545870000000003 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746331+0000", 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7805000000000005e-05 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746420+0000", 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8721000000000007e-05 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40956 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819640959000001, 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038823347000000001, 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40956 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:16.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707711+0000", 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1659999999999999e-06 2024-04-24T18:08:16.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707714+0000", 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.937e-06 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707716+0000", 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9920000000000002e-06 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746435+0000", 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718869000000003 2024-04-24T18:08:16.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746482+0000", 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6882999999999998e-05 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746533+0000", 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1711e-05 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40957 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819632684999998, 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039015052000000001, 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40957 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:16.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707721+0000", 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3110000000000001e-06 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707725+0000", 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2669999999999998e-06 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707727+0000", 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746563+0000", 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038836751000000003 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746641+0000", 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7900999999999998e-05 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746733+0000", 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2015999999999994e-05 2024-04-24T18:08:16.598 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40958 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2048000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819621991999998, 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039110399999999997, 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40958 2024-04-24T18:08:16.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707730+0000", 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2640000000000001e-06 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707733+0000", 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9170000000000002e-06 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707735+0000", 2024-04-24T18:08:16.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746744+0000", 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039008958000000003 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746789+0000", 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5019999999999999e-05 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746839+0000", 2024-04-24T18:08:16.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0531999999999997e-05 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40959 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.819613819000001, 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039290798000000002, 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:16.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40959 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707738+0000", 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.24e-06 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707742+0000", 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6200000000000001e-06 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:16.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707744+0000", 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.615e-06 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746859+0000", 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039115095000000003 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746938+0000", 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:16.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.747028+0000", 2024-04-24T18:08:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9742999999999998e-05 2024-04-24T18:08:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:16.607 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:16.607 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:16.608 INFO:teuthology.orchestra.run.smithi049.stdout:2881 writesame 0xb2dd0e1 thru 0xb2e7548 data_size 0x4(0xa468 bytes) 2024-04-24T18:08:16.608 INFO:teuthology.orchestra.run.smithi049.stdout:2882 writesame 0x16104eb thru 0x161ff09 data_size 0x1(0xfa1f bytes) 2024-04-24T18:08:16.609 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:2883 write 0x85a391b thru 0x85b1756 (0xde3c bytes) 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:2890 trunc from 0xb2e7549 to 0xd175f6f 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:2891 writesame 0x3b68521 thru 0x3b6a78c data_size 0x2(0x226c bytes) 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:2893 write 0xc6c4a4d thru 0xc6d3e86 (0xf43a bytes) 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:2894 write 0x990a409 thru 0x990b0be (0xcb6 bytes) 2024-04-24T18:08:16.611 INFO:teuthology.orchestra.run.smithi049.stdout:2895 punch from 0x89d9a8 to 0x8a7d47, (0xa39f bytes) 2024-04-24T18:08:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:2898 writesame 0xd0eaa27 thru 0xd0f1cb6 data_size 0x8d(0x7290 bytes) 2024-04-24T18:08:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:2899 read 0x37f7065 thru 0x3802171 (0xb10d bytes) 2024-04-24T18:08:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:2900 read 0x16272df thru 0x162ebbc (0x78de bytes) 2024-04-24T18:08:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:2901 read 0x5b4fc15 thru 0x5b5d890 (0xdc7c bytes) 2024-04-24T18:08:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:2902 writesame 0xc061c6f thru 0xc06bafd data_size 0x1(0x9e8f bytes) 2024-04-24T18:08:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:2903 write 0x6a3cd1b thru 0x6a48fa9 (0xc28f bytes) 2024-04-24T18:08:16.612 INFO:teuthology.orchestra.run.smithi049.stdout:2904 write 0xd25a306 thru 0xd2673b6 (0xd0b1 bytes) 2024-04-24T18:08:16.612 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:16.616+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:16.614 INFO:teuthology.orchestra.run.smithi049.stdout:2905 punch from 0x8357273 to 0x8360890, (0x961d bytes) 2024-04-24T18:08:16.614 INFO:teuthology.orchestra.run.smithi049.stdout:2906 read 0x6d10f1f thru 0x6d16b81 (0x5c63 bytes) 2024-04-24T18:08:16.614 INFO:teuthology.orchestra.run.smithi049.stdout:2907 clone 24 order 20 su 262144 sc 2 2024-04-24T18:08:16.713 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:16.721+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:16.747 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:16.748 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:16.748 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:16.749 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:16.764 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:16.784 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone23 from 0xd2673b7 (overlap 0xbfd106) to 0x132e80 2024-04-24T18:08:16.833 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:16.841+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:16.863 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #22, image image_client.0-clone22 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent23 2024-04-24T18:08:16.877 INFO:teuthology.orchestra.run.smithi049.stdout:2908 trunc from 0xd2673b7 to 0xae4c62b 2024-04-24T18:08:16.899 INFO:teuthology.orchestra.run.smithi049.stdout:2910 writesame 0xe7356da thru 0xe73d039 data_size 0x8(0x7960 bytes) 2024-04-24T18:08:16.899 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.899 INFO:teuthology.orchestra.run.smithi049.stdout:2915 read 0x78038eb thru 0x780d97e (0xa094 bytes) 2024-04-24T18:08:16.899 INFO:teuthology.orchestra.run.smithi049.stdout:2916 read 0xd5f50d thru 0xd66c35 (0x7729 bytes) 2024-04-24T18:08:16.899 INFO:teuthology.orchestra.run.smithi049.stdout:2917 write 0x3c62917 thru 0x3c64960 (0x204a bytes) 2024-04-24T18:08:16.902 INFO:teuthology.orchestra.run.smithi049.stdout:2918 read 0x289b6ec thru 0x289dfc0 (0x28d5 bytes) 2024-04-24T18:08:16.902 INFO:teuthology.orchestra.run.smithi049.stdout:2919 writesame 0x2dd1ea8 thru 0x2ddf9a3 data_size 0xa(0xdafc bytes) 2024-04-24T18:08:16.907 INFO:teuthology.orchestra.run.smithi049.stdout:2920 writesame 0xa3c39b6 thru 0xa3c7ae4 data_size 0x25(0x412f bytes) 2024-04-24T18:08:16.911 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.911 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.912 INFO:teuthology.orchestra.run.smithi049.stdout:2925 write 0x8bf8bb8 thru 0x8bfdad6 (0x4f1f bytes) 2024-04-24T18:08:16.914 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:16.923+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:16.915 INFO:teuthology.orchestra.run.smithi049.stdout:2926 punch from 0xa9919b3 to 0xa9984f0, (0x6b3d bytes) 2024-04-24T18:08:16.915 INFO:teuthology.orchestra.run.smithi049.stdout:2927 read 0x6a8d46f thru 0x6a90253 (0x2de5 bytes) 2024-04-24T18:08:16.915 INFO:teuthology.orchestra.run.smithi049.stdout:2929 write 0xda0b1e9 thru 0xda0e24c (0x3064 bytes) 2024-04-24T18:08:16.918 INFO:teuthology.orchestra.run.smithi049.stdout:2930 write 0x2aa924d thru 0x2ab462e (0xb3e2 bytes) 2024-04-24T18:08:16.921 INFO:teuthology.orchestra.run.smithi049.stdout:2932 write 0x848aa3f thru 0x849821e (0xd7e0 bytes) 2024-04-24T18:08:16.926 INFO:teuthology.orchestra.run.smithi049.stdout:2933 read 0xe3342cc thru 0xe33dbb4 (0x98e9 bytes) 2024-04-24T18:08:16.926 INFO:teuthology.orchestra.run.smithi049.stdout:2934 read 0x9969484 thru 0x9978fa6 (0xfb23 bytes) 2024-04-24T18:08:16.927 INFO:teuthology.orchestra.run.smithi049.stdout:2937 read 0x954a220 thru 0x955649f (0xc280 bytes) 2024-04-24T18:08:16.927 INFO:teuthology.orchestra.run.smithi049.stdout:2938 read 0x6df92ed thru 0x6dfbc95 (0x29a9 bytes) 2024-04-24T18:08:16.928 INFO:teuthology.orchestra.run.smithi049.stdout:2941 read 0xa64ea23 thru 0xa65b6b0 (0xcc8e bytes) 2024-04-24T18:08:16.931 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:16.931 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:16.931 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:16.931 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:16.932 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:16.938 INFO:teuthology.orchestra.run.smithi049.stdout:2944 trunc from 0xe73d03a to 0x4101745 2024-04-24T18:08:16.948 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:16.952 INFO:teuthology.orchestra.run.smithi049.stdout:2946 punch from 0x310fb64 to 0x311c974, (0xce10 bytes) 2024-04-24T18:08:16.952 INFO:teuthology.orchestra.run.smithi049.stdout:2947 write 0x3deb588 thru 0x3df8a9f (0xd518 bytes) 2024-04-24T18:08:16.954 INFO:teuthology.orchestra.run.smithi049.stdout:2948 read 0x1513452 thru 0x152323e (0xfded bytes) 2024-04-24T18:08:16.954 INFO:teuthology.orchestra.run.smithi049.stdout:2949 read 0xb7a6ba thru 0xb7c919 (0x2260 bytes) 2024-04-24T18:08:16.954 INFO:teuthology.orchestra.run.smithi049.stdout:2950 punch from 0x3acd1c to 0x3bca24, (0xfd08 bytes) 2024-04-24T18:08:16.962 INFO:teuthology.orchestra.run.smithi049.stdout:2951 write 0x654993e thru 0x654cbb5 (0x3278 bytes) 2024-04-24T18:08:16.965 INFO:teuthology.orchestra.run.smithi049.stdout:2953 write 0x5322f78 thru 0x532addf (0x7e68 bytes) 2024-04-24T18:08:16.968 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.968 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:16.968 INFO:teuthology.orchestra.run.smithi049.stdout:2956 read 0x192751c thru 0x192ece3 (0x77c8 bytes) 2024-04-24T18:08:16.977 INFO:teuthology.orchestra.run.smithi049.stdout:2958 write 0x8ffcfd1 thru 0x9007361 (0xa391 bytes) 2024-04-24T18:08:16.980 INFO:teuthology.orchestra.run.smithi049.stdout:2959 clone 25 order 20 su 16384 sc 8 2024-04-24T18:08:17.014 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:17.023+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:17.084 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:17.084 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9544149710000003, 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9544004309999998, 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:17.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:08:17.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:08:17.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9543855800000003, 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:17.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:08:17.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:08:17.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9543704310000001, 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.097 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9543556449999997, 2024-04-24T18:08:17.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9543410559999996, 2024-04-24T18:08:17.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.109 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9543182310000002, 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:08:17.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.954302577, 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9542874550000002, 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:08:17.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.121 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9542716389999999, 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.122 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9542566370000003, 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.126 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:08:17.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9542417729999997, 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:08:17.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:17.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:08:17.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9542206560000004, 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9542054259999997, 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.139 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9541906889999998, 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9490894789999995, 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:17.151 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9490536330000001, 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:17.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:08:17.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:08:17.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.155 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9490336260000003, 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:08:17.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:08:17.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.947212435, 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:08:17.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.9471871549999999, 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:08:17.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:08:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.169 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.169 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:17.171 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:17.171 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:17.179+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:17.272 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:17.280+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:17.340 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:17.340 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:17.340 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:17.340 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:17.350 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:17.395 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:17.404+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:17.474 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:17.483+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:17.479 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:17.479 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:17.479 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:17.479 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:17.479 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:17.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:17.577 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:17.585+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:17.659 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4880020749999998, 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4683629490000003, 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4683312270000002, 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4683078099999998, 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4682777370000002, 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4672325900000001, 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4672153589999999, 2024-04-24T18:08:17.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:17.688 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4671983610000003, 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:17.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:17.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4671796800000001, 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:17.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4671622119999999, 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.697 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4671460879999998, 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.701 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.467114982, 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.705 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:17.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4670957810000003, 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:17.709 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:17.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4670747390000001, 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:17.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4663197910000001, 2024-04-24T18:08:17.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:17.721 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4662932890000002, 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:17.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.725 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4662757590000002, 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:17.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:17.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:17.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.464812599, 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4529739509999997, 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.734 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:17.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:17.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 5.4529576080000002, 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:17.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:17.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.740 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:17.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:17.742 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:17.743 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:08:17.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:17.745 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:17.752+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:17.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:17.795 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone24 from 0x9007362 (overlap 0x4101745) to 0x0 2024-04-24T18:08:17.844 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:17.852+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:17.892 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:17.893 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:17.903 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:17.944 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #23, image image_client.0-clone23 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent24 2024-04-24T18:08:17.944 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:17.953+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:17.950 INFO:teuthology.orchestra.run.smithi049.stdout:2960 read 0x3a5c320 thru 0x3a5c515 (0x1f6 bytes) 2024-04-24T18:08:17.951 INFO:teuthology.orchestra.run.smithi049.stdout:2961 read 0x6ef030a thru 0x6efd156 (0xce4d bytes) 2024-04-24T18:08:17.953 INFO:teuthology.orchestra.run.smithi049.stdout:2962 trunc from 0x9007362 to 0x9ef4655 2024-04-24T18:08:17.962 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:17.962 INFO:teuthology.orchestra.run.smithi049.stdout:2965 clone 26 order 24 su 4194304 sc 9 2024-04-24T18:08:17.969 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:17.989 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:18.006 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:18.025 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:18.044 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:18.045 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:18.054+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:18.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:18.066 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:18.066 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:18.077 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:18.081 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:18.100 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:18.101 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:18.102 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:18.102 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:18.102 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:18.102 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:18.102 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:18.102 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:18.102 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:18.153 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:18.162+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:18.201 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:18.201 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:18.211 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:18.254 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:18.263+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:18.335 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:18.336 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:18.346 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:18.348 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:18.357+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:18.449 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:18.458+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:18.510 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:18.510 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:18.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:18.550 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:18.558+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:18.651 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:18.659+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:18.682 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:18.683 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:18.693 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:18.716 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:08:18.752 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:18.760+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:18.805 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone25 from 0x9ef4655 (overlap 0x9007362) to 0x2b355d0 2024-04-24T18:08:18.823 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:18.823 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:18.833 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:08:18.853 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:18.861+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:18.954 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:18.963+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:19.025 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:19.025 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:19.035 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:08:19.060 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #24, image image_client.0-clone24 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent25 2024-04-24T18:08:19.061 INFO:teuthology.orchestra.run.smithi049.stdout:2969 punch from 0x42715c8 to 0x4277c5f, (0x6697 bytes) 2024-04-24T18:08:19.065 INFO:teuthology.orchestra.run.smithi049.stdout:2970 read 0x7597d82 thru 0x75a798d (0xfc0c bytes) 2024-04-24T18:08:19.067 INFO:teuthology.orchestra.run.smithi049.stdout:2971 trunc from 0x9ef4655 to 0x81e5890 2024-04-24T18:08:19.076 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:19.084+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:19.156 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:19.164+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:19.159 INFO:teuthology.orchestra.run.smithi049.stdout:2972 write 0x6afff18 thru 0x6b0bda3 (0xbe8c bytes) 2024-04-24T18:08:19.167 INFO:teuthology.orchestra.run.smithi049.stdout:2975 writesame 0x999d8ea thru 0x99a30f6 data_size 0x1(0x580d bytes) 2024-04-24T18:08:19.167 INFO:teuthology.orchestra.run.smithi049.stdout:2976 read 0x988f148 thru 0x98969be (0x7877 bytes) 2024-04-24T18:08:19.167 INFO:teuthology.orchestra.run.smithi049.stdout:2977 punch from 0x44fe0e to 0x45f955, (0xfb47 bytes) 2024-04-24T18:08:19.168 INFO:teuthology.orchestra.run.smithi049.stdout:2978 read 0x6c661a3 thru 0x6c6c4ca (0x6328 bytes) 2024-04-24T18:08:19.168 INFO:teuthology.orchestra.run.smithi049.stdout:2979 punch from 0x96e5f3e to 0x96f216d, (0xc22f bytes) 2024-04-24T18:08:19.168 INFO:teuthology.orchestra.run.smithi049.stdout:2980 write 0x7e704f5 thru 0x7e794a2 (0x8fae bytes) 2024-04-24T18:08:19.204 INFO:teuthology.orchestra.run.smithi049.stdout:2981 punch from 0x131f1b8 to 0x1326f2d, (0x7d75 bytes) 2024-04-24T18:08:19.204 INFO:teuthology.orchestra.run.smithi049.stdout:2982 read 0x98e971f thru 0x98eaf60 (0x1842 bytes) 2024-04-24T18:08:19.204 INFO:teuthology.orchestra.run.smithi049.stdout:2984 write 0x413ba0e thru 0x414b7b9 (0xfdac bytes) 2024-04-24T18:08:19.206 INFO:teuthology.orchestra.run.smithi049.stdout:2986 read 0x82eb8a9 thru 0x82f48dd (0x9035 bytes) 2024-04-24T18:08:19.206 INFO:teuthology.orchestra.run.smithi049.stdout:2987 read 0x67d2ff thru 0x68625a (0x8f5c bytes) 2024-04-24T18:08:19.211 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:19.211 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:19.214 INFO:teuthology.orchestra.run.smithi049.stdout:2988 writesame 0xafd9f3c thru 0xafe1d85 data_size 0x6a(0x7e4a bytes) 2024-04-24T18:08:19.214 INFO:teuthology.orchestra.run.smithi049.stdout:2989 write 0x2c202f5 thru 0x2c22809 (0x2515 bytes) 2024-04-24T18:08:19.225 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:19.258 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:19.265+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:19.270 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:19.270 INFO:teuthology.orchestra.run.smithi049.stdout:2991 read 0x674d65b thru 0x6756be7 (0x958d bytes) 2024-04-24T18:08:19.270 INFO:teuthology.orchestra.run.smithi049.stdout:2992 writesame 0x5b3a59c thru 0x5b3da27 data_size 0x4c(0x348c bytes) 2024-04-24T18:08:19.271 INFO:teuthology.orchestra.run.smithi049.stdout:2994 write 0x2a6697f thru 0x2a72474 (0xbaf6 bytes) 2024-04-24T18:08:19.359 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:19.367+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:19.362 INFO:teuthology.orchestra.run.smithi049.stdout:2995 trunc from 0xafe1d86 to 0x3f53c68 2024-04-24T18:08:19.369 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:19.370 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:19.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:19.436 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:19.446 INFO:teuthology.orchestra.run.smithi049.stdout:2997 write 0x8e75a3d thru 0x8e76dca (0x138e bytes) 2024-04-24T18:08:19.448 INFO:teuthology.orchestra.run.smithi049.stdout:2998 write 0x7e1b554 thru 0x7e1bb88 (0x635 bytes) 2024-04-24T18:08:19.449 INFO:teuthology.orchestra.run.smithi049.stdout:2999 write 0x1fb419e thru 0x1fb664d (0x24b0 bytes) 2024-04-24T18:08:19.450 INFO:teuthology.orchestra.run.smithi049.stdout:3004 punch from 0x835ae3d to 0x8360fef, (0x61b2 bytes) 2024-04-24T18:08:19.451 INFO:teuthology.orchestra.run.smithi049.stdout:3007 read 0x64e41e2 thru 0x64f1fd7 (0xddf6 bytes) 2024-04-24T18:08:19.451 INFO:teuthology.orchestra.run.smithi049.stdout:3008 read 0x37aa00e thru 0x37b638b (0xc37e bytes) 2024-04-24T18:08:19.460 INFO:teuthology.orchestra.run.smithi049.stdout:3009 trunc from 0x8e76dcb to 0x177ee50 2024-04-24T18:08:19.460 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:19.468+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:19.473 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:19.473 INFO:teuthology.orchestra.run.smithi049.stdout:3015 punch from 0x13c6669 to 0x13cdf17, (0x78ae bytes) 2024-04-24T18:08:19.473 INFO:teuthology.orchestra.run.smithi049.stdout:3016 read 0x8b6700 thru 0x8bd8fe (0x71ff bytes) 2024-04-24T18:08:19.474 INFO:teuthology.orchestra.run.smithi049.stdout:3018 read 0x122c003 thru 0x122ce2c (0xe2a bytes) 2024-04-24T18:08:19.474 INFO:teuthology.orchestra.run.smithi049.stdout:3019 punch from 0x772ac0 to 0x77feb5, (0xd3f5 bytes) 2024-04-24T18:08:19.475 INFO:teuthology.orchestra.run.smithi049.stdout:3020 trunc from 0x177ee50 to 0x326f923 2024-04-24T18:08:19.489 INFO:teuthology.orchestra.run.smithi049.stdout:3022 writesame 0x4a9ffdb thru 0x4aa5d69 data_size 0x1(0x5d8f bytes) 2024-04-24T18:08:19.501 INFO:teuthology.orchestra.run.smithi049.stdout:3024 write 0x826125f thru 0x8261313 (0xb5 bytes) 2024-04-24T18:08:19.501 INFO:teuthology.orchestra.run.smithi049.stdout:3026 read 0x14aa4be thru 0x14afdaa (0x58ed bytes) 2024-04-24T18:08:19.502 INFO:teuthology.orchestra.run.smithi049.stdout:3027 read 0x59c29c3 thru 0x59ce6e9 (0xbd27 bytes) 2024-04-24T18:08:19.506 INFO:teuthology.orchestra.run.smithi049.stdout:3028 trunc from 0x8261314 to 0x44ff50d 2024-04-24T18:08:19.519 INFO:teuthology.orchestra.run.smithi049.stdout:3029 trunc from 0x44ff50d to 0x9f42982 2024-04-24T18:08:19.520 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:19.520 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:19.524 INFO:teuthology.orchestra.run.smithi049.stdout:3031 read 0x31d069a thru 0x31d5c1e (0x5585 bytes) 2024-04-24T18:08:19.525 INFO:teuthology.orchestra.run.smithi049.stdout:3032 writesame 0x12c3454 thru 0x12cb416 data_size 0x1(0x7fc3 bytes) 2024-04-24T18:08:19.531 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:19.546 INFO:teuthology.orchestra.run.smithi049.stdout:3033 write 0xae726fc thru 0xae78cfb (0x6600 bytes) 2024-04-24T18:08:19.549 INFO:teuthology.orchestra.run.smithi049.stdout:3034 writesame 0x1119a83 thru 0x11272c5 data_size 0xb(0xd843 bytes) 2024-04-24T18:08:19.552 INFO:teuthology.orchestra.run.smithi049.stdout:3035 read 0x6be8e8b thru 0x6bef26b (0x63e1 bytes) 2024-04-24T18:08:19.552 INFO:teuthology.orchestra.run.smithi049.stdout:3036 read 0x17e922a thru 0x17f2dc1 (0x9b98 bytes) 2024-04-24T18:08:19.553 INFO:teuthology.orchestra.run.smithi049.stdout:3037 read 0x99ab248 thru 0x99b85f3 (0xd3ac bytes) 2024-04-24T18:08:19.553 INFO:teuthology.orchestra.run.smithi049.stdout:3038 read 0x96964ff thru 0x969cfd1 (0x6ad3 bytes) 2024-04-24T18:08:19.553 INFO:teuthology.orchestra.run.smithi049.stdout:3041 read 0x91bc380 thru 0x91c5637 (0x92b8 bytes) 2024-04-24T18:08:19.562 INFO:teuthology.orchestra.run.smithi049.stdout:3042 writesame 0xcd7a341 thru 0xcd88e06 data_size 0xa2(0xeac6 bytes) 2024-04-24T18:08:19.562 INFO:teuthology.orchestra.run.smithi049.stdout:3044 punch from 0xbd1b5bc to 0xbd29d7c, (0xe7c0 bytes) 2024-04-24T18:08:19.562 INFO:teuthology.orchestra.run.smithi049.stdout:3046 trunc from 0xcd88e07 to 0xdb2c447 2024-04-24T18:08:19.568 INFO:teuthology.orchestra.run.smithi049.stdout:3047 writesame 0xd7f014f thru 0xd7fffdc data_size 0x3(0xfe8e bytes) 2024-04-24T18:08:19.574 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:19.583+0000 7f3a2b730700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 43495) UID: 0 2024-04-24T18:08:19.579 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:19.579 INFO:teuthology.orchestra.run.smithi049.stdout:3049 write 0xa957b43 thru 0xa95e00f (0x64cd bytes) 2024-04-24T18:08:19.582 INFO:teuthology.orchestra.run.smithi049.stdout:3050 punch from 0xc3b52a1 to 0xc3b5513, (0x272 bytes) 2024-04-24T18:08:19.582 INFO:teuthology.orchestra.run.smithi049.stdout:3051 read 0x7f55d0a thru 0x7f6006e (0xa365 bytes) 2024-04-24T18:08:19.583 INFO:teuthology.orchestra.run.smithi049.stdout:3052 punch from 0x89dd4f6 to 0x89df26e, (0x1d78 bytes) 2024-04-24T18:08:19.583 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:19.583 INFO:teuthology.orchestra.run.smithi049.stdout:3054 writesame 0x992dac5 thru 0x993924f data_size 0x1(0xb78b bytes) 2024-04-24T18:08:19.604 INFO:teuthology.orchestra.run.smithi049.stdout:3055 write 0x2036e5f thru 0x2040969 (0x9b0b bytes) 2024-04-24T18:08:19.607 INFO:teuthology.orchestra.run.smithi049.stdout:3056 punch from 0x53e74a8 to 0x53f26cc, (0xb224 bytes) 2024-04-24T18:08:19.607 INFO:teuthology.orchestra.run.smithi049.stdout:3057 write 0xfa0904 thru 0xfa6f88 (0x6685 bytes) 2024-04-24T18:08:19.622 INFO:teuthology.orchestra.run.smithi049.stdout:3058 trunc from 0xdb2c447 to 0x176b4e9 2024-04-24T18:08:19.654 INFO:teuthology.orchestra.run.smithi049.stdout:3059 write 0xa1d4103 thru 0xa1deac4 (0xa9c2 bytes) 2024-04-24T18:08:19.659 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:19.659 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:19.661 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:19.669+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:19.666 INFO:teuthology.orchestra.run.smithi049.stdout:3061 trunc from 0xa1deac5 to 0x1a73d9d 2024-04-24T18:08:19.669 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:19.698 INFO:teuthology.orchestra.run.smithi049.stdout:3062 writesame 0xee46977 thru 0xee4d7bf data_size 0x3(0x6e49 bytes) 2024-04-24T18:08:19.711 INFO:teuthology.orchestra.run.smithi049.stdout:3064 trunc from 0xee4d7c0 to 0x2e434a0 2024-04-24T18:08:19.717 INFO:teuthology.orchestra.run.smithi049.stdout:3065 punch from 0x6bb184 to 0x6ca78a, (0xf606 bytes) 2024-04-24T18:08:19.735 INFO:teuthology.orchestra.run.smithi049.stdout:3066 writesame 0xdb7a445 thru 0xdb81647 data_size 0x45(0x7203 bytes) 2024-04-24T18:08:19.747 INFO:teuthology.orchestra.run.smithi049.stdout:3067 trunc from 0xdb81648 to 0x2e4b9c6 2024-04-24T18:08:19.755 INFO:teuthology.orchestra.run.smithi049.stdout:3068 read 0x17ca66a thru 0x17d100b (0x69a2 bytes) 2024-04-24T18:08:19.755 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:19.755 INFO:teuthology.orchestra.run.smithi049.stdout:truncating to largest ever: 0xedc48f0 2024-04-24T18:08:19.762 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:19.770+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:19.769 INFO:teuthology.orchestra.run.smithi049.stdout:3071 trunc from 0x2e4b9c6 to 0xedc48f0 2024-04-24T18:08:19.773 INFO:teuthology.orchestra.run.smithi049.stdout:3072 write 0xf6563d thru 0xf68137 (0x2afb bytes) 2024-04-24T18:08:19.781 INFO:teuthology.orchestra.run.smithi049.stdout:3075 trunc from 0xedc48f0 to 0x9366bb7 2024-04-24T18:08:19.787 INFO:teuthology.orchestra.run.smithi049.stdout:3076 clone 27 order 22 su 32768 sc 12 2024-04-24T18:08:19.819 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:19.819 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:19.829 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:19.864 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:19.872+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:19.965 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:19.973+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:19.997 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:19.997 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:20.007 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:20.066 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:20.074+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:20.113 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:20.136 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:20.136 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:20.147 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:20.188 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:20.196+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:20.259 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:20.268 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:20.276+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:20.277 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:20.296 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:20.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:20.321 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:20.321 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:20.321 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:20.321 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:20.332 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:20.336 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:20.355 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:20.369 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:20.378+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:20.375 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:20.394 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:20.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:20.465 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:20.465 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:20.465 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:20.465 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:20.465 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:20.470 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:20.479+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:20.475 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:20.572 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:20.580+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:20.604 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40940 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1458176~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904447192999999, 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036446457000000002, 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.605 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40940 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707547+0000", 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707548+0000", 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.251e-06 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707551+0000", 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0019999999999998e-06 2024-04-24T18:08:20.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707554+0000", 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0490000000000002e-06 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743860+0000", 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036306756000000003 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743915+0000", 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4423999999999999e-05 2024-04-24T18:08:20.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.743994+0000", 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9207000000000002e-05 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40941 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1490944~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904413642000002, 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036634427999999997, 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.609 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40941 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707581+0000", 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707583+0000", 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2089999999999999e-06 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707587+0000", 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1670000000000001e-06 2024-04-24T18:08:20.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707590+0000", 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.554e-06 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744033+0000", 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036443542000000002 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744112+0000", 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8578999999999994e-05 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744215+0000", 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000103607 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40942 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1523712~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904402596000001, 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036756906999999998, 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.613 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40942 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707592+0000", 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707594+0000", 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7090000000000001e-06 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707596+0000", 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:20.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707598+0000", 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9149999999999999e-06 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744245+0000", 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036646570000000003 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744295+0000", 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0343000000000002e-05 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744349+0000", 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3661e-05 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40943 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1556480~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904393917, 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036956462000000002, 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40943 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707601+0000", 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707600+0000", 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707602+0000", 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.215e-06 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707605+0000", 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0429999999999999e-06 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707607+0000", 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9089999999999998e-06 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744380+0000", 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036773477999999998 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744464+0000", 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4048000000000006e-05 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744557+0000", 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3043999999999999e-05 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40944 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1589248~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:20.621 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904385916999999, 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037054089999999998, 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40944 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:20.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707609+0000", 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707610+0000", 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1319999999999999e-06 2024-04-24T18:08:20.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707613+0000", 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707614+0000", 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8819999999999999e-06 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744569+0000", 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.036954539000000002 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744613+0000", 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3671999999999998e-05 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744663+0000", 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0145000000000002e-05 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40945 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1622016~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:20.625 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904378016999999, 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037248315999999997, 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40945 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707617+0000", 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707616+0000", 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707620+0000", 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1999999999999999e-06 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707623+0000", 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1669999999999999e-06 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707625+0000", 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1110000000000002e-06 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744693+0000", 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037068564999999998 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744773+0000", 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9172999999999996e-05 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744865+0000", 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2305000000000005e-05 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40946 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1654784~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904367474000001, 2024-04-24T18:08:20.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037340299, 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40946 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707627+0000", 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707628+0000", 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2699999999999999e-06 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707631+0000", 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9519999999999999e-06 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707633+0000", 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.8619999999999999e-06 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744876+0000", 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037242550999999999 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744918+0000", 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2546000000000002e-05 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744967+0000", 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9320999999999998e-05 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40947 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1687552~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904359417999999, 2024-04-24T18:08:20.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037537860999999999, 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40947 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707635+0000", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707636+0000", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.186e-06 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707639+0000", 2024-04-24T18:08:20.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9179999999999998e-06 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707641+0000", 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.86e-06 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.744997+0000", 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037356364000000003 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745076+0000", 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8355000000000005e-05 2024-04-24T18:08:20.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745173+0000", 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7379000000000004e-05 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40948 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1720320~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904351494, 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037655182000000002, 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.637 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40948 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:20.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707643+0000", 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707644+0000", 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.269e-06 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707647+0000", 2024-04-24T18:08:20.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9359999999999999e-06 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707649+0000", 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.815e-06 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745200+0000", 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037551189999999998 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745246+0000", 2024-04-24T18:08:20.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6214999999999997e-05 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745298+0000", 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1960999999999999e-05 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40949 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1753088~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904343465, 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037846454000000002, 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40949 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:20.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707651+0000", 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707652+0000", 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.13e-06 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707655+0000", 2024-04-24T18:08:20.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707658+0000", 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5770000000000001e-06 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745329+0000", 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037671351999999998 2024-04-24T18:08:20.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745408+0000", 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745498+0000", 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0043000000000005e-05 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40950 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1785856~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904334785, 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037946947000000002, 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40950 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707660+0000", 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707661+0000", 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.15e-06 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707664+0000", 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.965e-06 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707666+0000", 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3719999999999999e-06 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745508+0000", 2024-04-24T18:08:20.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037841977999999998 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745555+0000", 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7098999999999997e-05 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745607+0000", 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1591999999999997e-05 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40951 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1818624~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904326305000001, 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038137931999999999, 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40951 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707668+0000", 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707670+0000", 2024-04-24T18:08:20.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.7010000000000001e-06 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707673+0000", 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9119999999999998e-06 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707675+0000", 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9429999999999999e-06 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745637+0000", 2024-04-24T18:08:20.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.037962379999999997 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745715+0000", 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7488000000000001e-05 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745806+0000", 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1723000000000003e-05 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40952 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1851392~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904317679000002, 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038236762000000001, 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40952 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:20.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707677+0000", 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707678+0000", 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.093e-06 2024-04-24T18:08:20.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707681+0000", 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.92e-06 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707683+0000", 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.951e-06 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745817+0000", 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038133854000000002 2024-04-24T18:08:20.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745862+0000", 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5027000000000001e-05 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745914+0000", 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2120999999999998e-05 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.658 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40953 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1884160~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904309646000002, 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038424458000000002, 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40953 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:20.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707685+0000", 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707686+0000", 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.567e-06 2024-04-24T18:08:20.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707689+0000", 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9220000000000001e-06 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707691+0000", 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9369999999999998e-06 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.745943+0000", 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038252047999999997 2024-04-24T18:08:20.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746018+0000", 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4938000000000007e-05 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746109+0000", 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1255000000000006e-05 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.662 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40954 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1916928~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904301137000001, 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038528139000000003, 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40954 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:20.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707693+0000", 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707694+0000", 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.065e-06 2024-04-24T18:08:20.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707697+0000", 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.881e-06 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707699+0000", 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.88e-06 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746120+0000", 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038420937000000002 2024-04-24T18:08:20.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746171+0000", 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0555e-05 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746222+0000", 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1025e-05 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40955 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1949696~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.666 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904293317, 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718855000000003, 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40955 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707701+0000", 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707702+0000", 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.212e-06 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707706+0000", 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.534e-06 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707708+0000", 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746254+0000", 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038545870000000003 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746331+0000", 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7805000000000005e-05 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746420+0000", 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8721000000000007e-05 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40956 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 1982464~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:20.670 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904284699000002, 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038823347000000001, 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40956 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707710+0000", 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707711+0000", 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1659999999999999e-06 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707714+0000", 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.937e-06 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707716+0000", 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9920000000000002e-06 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746435+0000", 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038718869000000003 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746482+0000", 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6882999999999998e-05 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746533+0000", 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1711e-05 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40957 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2015232~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.674 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904276424999999, 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039015052000000001, 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40957 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707718+0000", 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707721+0000", 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.3110000000000001e-06 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707725+0000", 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2669999999999998e-06 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707727+0000", 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746563+0000", 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.038836751000000003 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746641+0000", 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7900999999999998e-05 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746733+0000", 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2015999999999994e-05 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40958 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2048000~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904265731999999, 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039110399999999997, 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40958 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707729+0000", 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707730+0000", 2024-04-24T18:08:20.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2640000000000001e-06 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707733+0000", 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9170000000000002e-06 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707735+0000", 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.916e-06 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746744+0000", 2024-04-24T18:08:20.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039008958000000003 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746789+0000", 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.5019999999999999e-05 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746839+0000", 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0531999999999997e-05 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40959 4.f 4:f0e0de80:::rbd_data.1174b285a870.0000000000000007:c [read 2080768~8192] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e47)", 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.904257559000001, 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039290798000000002, 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40959 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:20.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707737+0000", 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967296 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707738+0000", 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.24e-06 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707742+0000", 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6200000000000001e-06 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.707744+0000", 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.615e-06 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:20.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746859+0000", 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.039115095000000003 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.746938+0000", 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8690999999999996e-05 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.747028+0000", 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9742999999999998e-05 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.686 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:20.687 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:20.687 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:20.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:20.688 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:20.696+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:20.727 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:08:20.727 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:08:20.727 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-04-24T18:08:20.728 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T18:08:20.728 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:08:20.729 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:08:20.765 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T18:08:20.765 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-04-24T18:08:20.766 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:08:20.766 DEBUG:teuthology.orchestra.run.smithi078:> type ceph-objectstore-tool 2024-04-24T18:08:20.788 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:20.796+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:20.825 INFO:teuthology.orchestra.run.smithi078.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-04-24T18:08:20.825 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-04-24T18:08:20.867 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:20.867 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:20.867 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:20.867 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:20.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:20.911 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:20.919+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:20.990 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:20.998+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:21.047 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:21.047 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:21.047 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:21.047 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:21.047 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:21.058 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:21.102 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:21.110+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0580634199999999, 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.188 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:21.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0580488799999994, 2024-04-24T18:08:21.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.196 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0580340289999999, 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.201 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0580188800000006, 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:21.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0580040939999993, 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0579895050000001, 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0579666799999998, 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.217 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0579510259999996, 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.218 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:08:21.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:08:21.221 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0579359040000007, 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:08:21.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:08:21.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:08:21.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0579200879999995, 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:08:21.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:08:21.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:08:21.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0579050859999999, 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.230 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:21.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:08:21.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:08:21.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0578902219999993, 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.234 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:08:21.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.057869105, 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.238 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0578538749999993, 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.242 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0578391380000003, 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.246 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:21.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0527379280000009, 2024-04-24T18:08:21.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:21.254 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0527020819999997, 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.258 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0526820749999999, 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:21.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:08:21.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0508608840000004, 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.263 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.0508356039999995, 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.267 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:21.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.271 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:21.273 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:21.273 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:21.281+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:21.373 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:21.381+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:21.443 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:21.443 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:21.443 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:21.443 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:21.453 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:21.463 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:08:21.471+0000 7f5158ec0340 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-04-24T18:08:21.494 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:21.502+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:21.576 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:21.583+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:21.622 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:21.622 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:21.622 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:21.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:21.716 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:21.724+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:21.735 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:08:21.787 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:21.795+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6357973650000002, 2024-04-24T18:08:21.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:21.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:21.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:21.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6161582390000007, 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6161265169999997, 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.816 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:21.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:21.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6161031000000001, 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:21.820 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:21.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:21.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:21.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6160730270000006, 2024-04-24T18:08:21.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.829 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6150278799999995, 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:21.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6150106490000002, 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:21.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:21.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:21.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.838 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6149936510000007, 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:21.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6149749700000005, 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:21.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:21.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:21.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6149575019999993, 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:21.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.852 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6149413779999993, 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:21.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6149102719999995, 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:21.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:21.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:21.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:21.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6148910710000006, 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6148700290000004, 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:21.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6141150809999996, 2024-04-24T18:08:21.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:21.873 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6140885790000006, 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.877 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6140710489999996, 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:21.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6126078889999995, 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:21.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6007692410000001, 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:21.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:21.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:21.890 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 9.6007528979999996, 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:21.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:21.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:21.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.895 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:21.895 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:21.895 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:21.896 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:21.897 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:21.905+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:21.997 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:22.005+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:22.022 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:22.023 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:22.023 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:22.033 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:22.099 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:22.107+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:22.158 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:22.159 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:22.168 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:22.212 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:22.221+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11780 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2666496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:22.301 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.971799943000001, 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042348956, 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11780 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:22.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335768+0000", 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1910000000000003e-06 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335774+0000", 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9089999999999997e-06 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335778+0000", 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9339999999999999e-06 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378028+0000", 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042250303000000003 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378072+0000", 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3909000000000003e-05 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378110+0000", 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8050000000000003e-05 2024-04-24T18:08:22.305 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11792 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.969787146999998, 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042313245999999999, 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11792 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337773+0000", 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337781+0000", 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3089999999999998e-06 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337787+0000", 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.285e-06 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337791+0000", 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.065e-06 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.379933+0000", 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042141913000000003 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380016+0000", 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2959000000000003e-05 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380087+0000", 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1038000000000007e-05 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11799 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2856960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.968549799000002, 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042339663999999999, 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.310 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11799 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339013+0000", 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339019+0000", 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339023+0000", 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7689999999999998e-06 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381202+0000", 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042179054000000001 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381285+0000", 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3299000000000003e-05 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381351+0000", 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5795000000000003e-05 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.968534019000003, 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.314 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:22.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:08:22.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:08:22.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.968518113000002, 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:08:22.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.968466145999997, 2024-04-24T18:08:22.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11805 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2920448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:22.326 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.967559446000003, 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042290415999999997, 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11805 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340003+0000", 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.514e-06 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340010+0000", 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340014+0000", 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.805e-06 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382196+0000", 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042182175000000002 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382236+0000", 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0265999999999997e-05 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382292+0000", 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5791000000000002e-05 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.330 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.967542672, 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.331 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:08:22.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:08:22.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.967526040000003, 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:22.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11808 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2957312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340690+0000", 2024-04-24T18:08:22.339 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.966870839999999, 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042195728000000002, 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11808 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340690+0000", 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340690+0000", 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340052+0000", 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999362 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340695+0000", 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00064299599999999995 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340705+0000", 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0369e-05 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340711+0000", 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0460000000000002e-06 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382725+0000", 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042013834999999999 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382812+0000", 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7158999999999997e-05 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382886+0000", 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3274000000000004e-05 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11809 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2965504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.966842587000002, 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042364339000000001, 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11809 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340721+0000", 2024-04-24T18:08:22.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1370000000000002e-06 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340727+0000", 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.533e-06 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340732+0000", 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5329999999999996e-06 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382924+0000", 2024-04-24T18:08:22.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042192039000000001 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383012+0000", 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7669000000000004e-05 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383083+0000", 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0988999999999994e-05 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.347 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.966824437, 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:22.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:22.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:08:22.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.966807934999999, 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:08:22.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11814 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3012608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.965913352999998, 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042354992000000001, 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.356 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11814 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341648+0000", 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.372e-06 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341652+0000", 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0690000000000002e-06 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341654+0000", 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0559999999999999e-06 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383852+0000", 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042198125000000003 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383934+0000", 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2021999999999995e-05 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384002+0000", 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8669000000000003e-05 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.965905069000002, 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.360 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:08:22.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:08:22.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.965897044000002, 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11819 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3065856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.964865334999999, 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042208806000000001, 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.368 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11819 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342698+0000", 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6879999999999999e-06 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342704+0000", 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0090000000000004e-06 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342708+0000", 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9029999999999997e-06 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384745+0000", 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042037243000000002 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384833+0000", 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7769999999999995e-05 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384904+0000", 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1632999999999997e-05 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11820 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3076096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:22.372 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.964848670999999, 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042328461999999997, 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11820 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:22.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342714+0000", 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342720+0000", 2024-04-24T18:08:22.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8479999999999997e-06 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342724+0000", 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6430000000000001e-06 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384944+0000", 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042220698000000001 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384992+0000", 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7441999999999997e-05 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385041+0000", 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8715999999999999e-05 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.964832901999998, 2024-04-24T18:08:22.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.381 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.964816933000002, 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:22.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:22.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:22.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:08:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:08:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:22.386 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:22.386 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:22.387 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:22.308+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:22.388 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:22.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:22.429 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:22.437+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:22.502 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:22.510+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:22.552 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:22.553 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:22.553 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:22.553 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:22.563 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:22.564 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:22.582 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:22.600 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:22.603 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:22.611+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:22.619 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:22.637 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:22.656 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:22.676 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:22.676 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:22.677 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:22.677 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:22.677 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:22.678 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:22.679 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:22.689 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:22.689 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:22.700 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:22.720 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:22.728+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:22.765 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone26 from 0x9366bb7 (overlap 0x176b4e9) to 0xe0d2f2 2024-04-24T18:08:22.805 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:22.813+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:22.834 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:22.834 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:22.845 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:22.887 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #25, image image_client.0-clone25 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent26 2024-04-24T18:08:22.907 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:22.916+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:22.996 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:22.996 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:23.007 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:23.008 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:23.015+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:23.104 INFO:teuthology.orchestra.run.smithi049.stdout:3077 write 0xcf8550d thru 0xcf8c701 (0x71f5 bytes) 2024-04-24T18:08:23.110 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:23.117+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:23.115 INFO:teuthology.orchestra.run.smithi049.stdout:3078 writesame 0xde0cfb1 thru 0xde1a39f data_size 0x5(0xd3ef bytes) 2024-04-24T18:08:23.116 INFO:teuthology.orchestra.run.smithi049.stdout:3079 write 0x8659580 thru 0x86645bd (0xb03e bytes) 2024-04-24T18:08:23.138 INFO:teuthology.orchestra.run.smithi049.stdout:3080 trunc from 0xde1a3a0 to 0x12d443d 2024-04-24T18:08:23.208 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:23.208 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:23.211 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:23.218+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:23.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:23.233 INFO:teuthology.orchestra.run.smithi049.stdout:3081 read 0x9b26ca thru 0x9c2417 (0xfd4e bytes) 2024-04-24T18:08:23.233 INFO:teuthology.orchestra.run.smithi049.stdout:3082 punch from 0x19b547 to 0x19c67a, (0x1133 bytes) 2024-04-24T18:08:23.248 INFO:teuthology.orchestra.run.smithi049.stdout:3083 writesame 0x8ab3f40 thru 0x8ac1c2c data_size 0x1(0xdced bytes) 2024-04-24T18:08:23.248 INFO:teuthology.orchestra.run.smithi049.stdout:3084 write 0x19a627f thru 0x19a67f2 (0x574 bytes) 2024-04-24T18:08:23.253 INFO:teuthology.orchestra.run.smithi049.stdout:3085 trunc from 0x8ac1c2d to 0xc2abc5c 2024-04-24T18:08:23.258 INFO:teuthology.orchestra.run.smithi049.stdout:3086 write 0x605ee31 thru 0x605eedf (0xaf bytes) 2024-04-24T18:08:23.261 INFO:teuthology.orchestra.run.smithi049.stdout:3087 punch from 0x44a34d8 to 0x44ad3c0, (0x9ee8 bytes) 2024-04-24T18:08:23.261 INFO:teuthology.orchestra.run.smithi049.stdout:3089 write 0x3c9ee66 thru 0x3c9fffe (0x1199 bytes) 2024-04-24T18:08:23.264 INFO:teuthology.orchestra.run.smithi049.stdout:3090 writesame 0x6623435 thru 0x662bcb6 data_size 0x2(0x8882 bytes) 2024-04-24T18:08:23.273 INFO:teuthology.orchestra.run.smithi049.stdout:3091 write 0x5d467c8 thru 0x5d4ac06 (0x443f bytes) 2024-04-24T18:08:23.280 INFO:teuthology.orchestra.run.smithi049.stdout:3092 trunc from 0xc2abc5c to 0x9722478 2024-04-24T18:08:23.287 INFO:teuthology.orchestra.run.smithi049.stdout:3093 write 0x93cb8a1 thru 0x93d4fc5 (0x9725 bytes) 2024-04-24T18:08:23.290 INFO:teuthology.orchestra.run.smithi049.stdout:3094 write 0x332b3d3 thru 0x33395d6 (0xe204 bytes) 2024-04-24T18:08:23.297 INFO:teuthology.orchestra.run.smithi049.stdout:3096 write 0x8e4f52f thru 0x8e5552f (0x6001 bytes) 2024-04-24T18:08:23.301 INFO:teuthology.orchestra.run.smithi049.stdout:3099 read 0x777986e thru 0x777c7bd (0x2f50 bytes) 2024-04-24T18:08:23.302 INFO:teuthology.orchestra.run.smithi049.stdout:3100 write 0x1854fb2 thru 0x185de82 (0x8ed1 bytes) 2024-04-24T18:08:23.309 INFO:teuthology.orchestra.run.smithi049.stdout:3101 writesame 0x994d4fa thru 0x9952ab0 data_size 0x1(0x55b7 bytes) 2024-04-24T18:08:23.309 INFO:teuthology.orchestra.run.smithi049.stdout:3102 read 0x429825b thru 0x42a0488 (0x822e bytes) 2024-04-24T18:08:23.309 INFO:teuthology.orchestra.run.smithi049.stdout:3103 read 0x94a608b thru 0x94aa4b0 (0x4426 bytes) 2024-04-24T18:08:23.309 INFO:teuthology.orchestra.run.smithi049.stdout:3104 write 0x42ea26 thru 0x43c53e (0xdb19 bytes) 2024-04-24T18:08:23.311 INFO:teuthology.orchestra.run.smithi049.stdout:3105 writesame 0x5c4f2da thru 0x5c5197d data_size 0x2(0x26a4 bytes) 2024-04-24T18:08:23.311 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:23.319+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:23.319 INFO:teuthology.orchestra.run.smithi049.stdout:3106 punch from 0x7a7f264 to 0x7a8a197, (0xaf33 bytes) 2024-04-24T18:08:23.319 INFO:teuthology.orchestra.run.smithi049.stdout:3108 write 0x5074bc5 thru 0x50787c6 (0x3c02 bytes) 2024-04-24T18:08:23.329 INFO:teuthology.orchestra.run.smithi049.stdout:3109 write 0xb90664b thru 0xb916248 (0xfbfe bytes) 2024-04-24T18:08:23.338 INFO:teuthology.orchestra.run.smithi049.stdout:3110 punch from 0x13e2b4d to 0x13e6a2b, (0x3ede bytes) 2024-04-24T18:08:23.340 INFO:teuthology.orchestra.run.smithi049.stdout:3111 trunc from 0xb916249 to 0xa50a323 2024-04-24T18:08:23.347 INFO:teuthology.orchestra.run.smithi049.stdout:3112 write 0x9fe2c70 thru 0x9fec185 (0x9516 bytes) 2024-04-24T18:08:23.349 INFO:teuthology.orchestra.run.smithi049.stdout:3113 read 0x4c614f9 thru 0x4c63385 (0x1e8d bytes) 2024-04-24T18:08:23.349 INFO:teuthology.orchestra.run.smithi049.stdout:3114 read 0x60727c2 thru 0x607f8d9 (0xd118 bytes) 2024-04-24T18:08:23.350 INFO:teuthology.orchestra.run.smithi049.stdout:3115 read 0x63a971b thru 0x63aa978 (0x125e bytes) 2024-04-24T18:08:23.350 INFO:teuthology.orchestra.run.smithi049.stdout:3116 writesame 0x8f37a9d thru 0x8f422e5 data_size 0x1(0xa849 bytes) 2024-04-24T18:08:23.350 INFO:teuthology.orchestra.run.smithi078.stderr:2024-04-24T18:08:23.358+0000 7f5158ec0340 -1 bluestore::NCB::__restore_allocator::No Valid allocation info on disk (empty file) 2024-04-24T18:08:23.359 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:23.359 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:23.369 INFO:teuthology.orchestra.run.smithi049.stdout:3118 read 0x506793b thru 0x506e726 (0x6dec bytes) 2024-04-24T18:08:23.369 INFO:teuthology.orchestra.run.smithi049.stdout:3120 punch from 0x9449227 to 0x94513e8, (0x81c1 bytes) 2024-04-24T18:08:23.369 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:23.369 INFO:teuthology.orchestra.run.smithi049.stdout:3122 punch from 0x33bd58 to 0x345ff9, (0xa2a1 bytes) 2024-04-24T18:08:23.370 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:23.377 INFO:teuthology.orchestra.run.smithi049.stdout:3123 write 0xcb33589 thru 0xcb3490c (0x1384 bytes) 2024-04-24T18:08:23.379 INFO:teuthology.orchestra.run.smithi049.stdout:3124 read 0xb4869e5 thru 0xb4935e1 (0xcbfd bytes) 2024-04-24T18:08:23.380 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:23.380 INFO:teuthology.orchestra.run.smithi049.stdout:3127 punch from 0x23978fb to 0x2399eea, (0x25ef bytes) 2024-04-24T18:08:23.385 INFO:teuthology.orchestra.run.smithi049.stdout:3128 write 0xd4b5207 thru 0xd4b9fd0 (0x4dca bytes) 2024-04-24T18:08:23.389 INFO:teuthology.orchestra.run.smithi049.stdout:3129 writesame 0xc6beca0 thru 0xc6c9649 data_size 0x7f(0xa9aa bytes) 2024-04-24T18:08:23.393 INFO:teuthology.orchestra.run.smithi049.stdout:3130 writesame 0x386da6b thru 0x3875656 data_size 0x67(0x7bec bytes) 2024-04-24T18:08:23.394 INFO:teuthology.orchestra.run.smithi049.stdout:3131 read 0xd3c7fe6 thru 0xd3d3dd8 (0xbdf3 bytes) 2024-04-24T18:08:23.395 INFO:teuthology.orchestra.run.smithi049.stdout:3134 punch from 0x8d4fb33 to 0x8d5c868, (0xcd35 bytes) 2024-04-24T18:08:23.395 INFO:teuthology.orchestra.run.smithi049.stdout:3135 write 0xb16774 thru 0xb23b91 (0xd41e bytes) 2024-04-24T18:08:23.397 INFO:teuthology.orchestra.run.smithi049.stdout:3136 write 0x44ff6f6 thru 0x4506042 (0x694d bytes) 2024-04-24T18:08:23.401 INFO:teuthology.orchestra.run.smithi049.stdout:3138 writesame 0x2aaeb09 thru 0x2ab6461 data_size 0x5f(0x7959 bytes) 2024-04-24T18:08:23.405 INFO:teuthology.orchestra.run.smithi049.stdout:3139 trunc from 0xd4b9fd1 to 0xa9b61c3 2024-04-24T18:08:23.413 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:23.420+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:23.418 INFO:teuthology.orchestra.run.smithi049.stdout:3140 write 0xa117ced thru 0xa11a3e2 (0x26f6 bytes) 2024-04-24T18:08:23.421 INFO:teuthology.orchestra.run.smithi049.stdout:3141 writesame 0x9e9a763 thru 0x9ea6db5 data_size 0x1(0xc653 bytes) 2024-04-24T18:08:23.440 INFO:teuthology.orchestra.run.smithi049.stdout:3142 read 0x2e67652 thru 0x2e73ec0 (0xc86f bytes) 2024-04-24T18:08:23.440 INFO:teuthology.orchestra.run.smithi049.stdout:3143 punch from 0x80de4e2 to 0x80e56ab, (0x71c9 bytes) 2024-04-24T18:08:23.444 INFO:teuthology.orchestra.run.smithi049.stdout:3144 trunc from 0xa9b61c3 to 0xd9b2cbf 2024-04-24T18:08:23.449 INFO:teuthology.orchestra.run.smithi049.stdout:3145 punch from 0x5ebeae9 to 0x5ecaa82, (0xbf99 bytes) 2024-04-24T18:08:23.449 INFO:teuthology.orchestra.run.smithi049.stdout:3146 punch from 0x6fd68e5 to 0x6fe049f, (0x9bba bytes) 2024-04-24T18:08:23.449 INFO:teuthology.orchestra.run.smithi049.stdout:3149 write 0x9b65d72 thru 0x9b6df4b (0x81da bytes) 2024-04-24T18:08:23.450 INFO:teuthology.orchestra.run.smithi049.stdout:3151 read 0x4822aa thru 0x486e3f (0x4b96 bytes) 2024-04-24T18:08:23.451 INFO:teuthology.orchestra.run.smithi049.stdout:3152 write 0x7373748 thru 0x737ff93 (0xc84c bytes) 2024-04-24T18:08:23.453 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:23.453 INFO:teuthology.orchestra.run.smithi049.stdout:3154 punch from 0x233ed35 to 0x234124c, (0x2517 bytes) 2024-04-24T18:08:23.453 INFO:teuthology.orchestra.run.smithi049.stdout:3155 write 0x73cbe83 thru 0x73d290a (0x6a88 bytes) 2024-04-24T18:08:23.456 INFO:teuthology.orchestra.run.smithi049.stdout:3156 writesame 0x16c1c98 thru 0x16d0a4c data_size 0x1(0xedb5 bytes) 2024-04-24T18:08:23.486 INFO:teuthology.orchestra.run.smithi049.stdout:3157 trunc from 0xd9b2cbf to 0x6302df9 2024-04-24T18:08:23.508 INFO:teuthology.orchestra.run.smithi049.stdout:3158 write 0x94a9ad8 thru 0x94b659f (0xcac8 bytes) 2024-04-24T18:08:23.511 INFO:teuthology.orchestra.run.smithi049.stdout:3160 read 0x6803f57 thru 0x681305c (0xf106 bytes) 2024-04-24T18:08:23.514 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:23.521+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:23.521 INFO:teuthology.orchestra.run.smithi049.stdout:3161 trunc from 0x94b65a0 to 0x2509e1c 2024-04-24T18:08:23.534 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:23.534 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:23.535 INFO:teuthology.orchestra.run.smithi049.stdout:3162 read 0x77384c thru 0x778dbc (0x5571 bytes) 2024-04-24T18:08:23.542 INFO:teuthology.orchestra.run.smithi049.stdout:3163 write 0x39ee4f9 thru 0x39efa33 (0x153b bytes) 2024-04-24T18:08:23.545 INFO:teuthology.orchestra.run.smithi049.stdout:3164 read 0x18e991e thru 0x18f5ace (0xc1b1 bytes) 2024-04-24T18:08:23.546 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:23.556 INFO:teuthology.orchestra.run.smithi049.stdout:3165 writesame 0x77d81f0 thru 0x77e54c2 data_size 0x1(0xd2d3 bytes) 2024-04-24T18:08:23.556 INFO:teuthology.orchestra.run.smithi049.stdout:3167 write 0x51c4405 thru 0x51d35d9 (0xf1d5 bytes) 2024-04-24T18:08:23.563 INFO:teuthology.orchestra.run.smithi049.stdout:3168 read 0x2d62137 thru 0x2d6c3b4 (0xa27e bytes) 2024-04-24T18:08:23.567 INFO:teuthology.orchestra.run.smithi049.stdout:3172 trunc from 0x77e54c3 to 0x4847acb 2024-04-24T18:08:23.573 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:23.573 INFO:teuthology.orchestra.run.smithi049.stdout:3174 read 0x232c7ed thru 0x2331600 (0x4e14 bytes) 2024-04-24T18:08:23.573 INFO:teuthology.orchestra.run.smithi049.stdout:3175 read 0x206247b thru 0x2069c3e (0x77c4 bytes) 2024-04-24T18:08:23.574 INFO:teuthology.orchestra.run.smithi049.stdout:3176 punch from 0x89804 to 0x994e2, (0xfcde bytes) 2024-04-24T18:08:23.574 INFO:teuthology.orchestra.run.smithi049.stdout:3178 punch from 0xa615c2 to 0xa6b196, (0x9bd4 bytes) 2024-04-24T18:08:23.574 INFO:teuthology.orchestra.run.smithi049.stdout:3181 read 0x2418895 thru 0x2419db4 (0x1520 bytes) 2024-04-24T18:08:23.584 INFO:teuthology.orchestra.run.smithi049.stdout:3182 writesame 0x7bfc5a2 thru 0x7bffef4 data_size 0x5(0x3953 bytes) 2024-04-24T18:08:23.584 INFO:teuthology.orchestra.run.smithi049.stdout:3183 write 0x702a20b thru 0x702d89b (0x3691 bytes) 2024-04-24T18:08:23.597 INFO:teuthology.orchestra.run.smithi049.stdout:3184 writesame 0xd700724 thru 0xd70a57f data_size 0xa(0x9e5c bytes) 2024-04-24T18:08:23.598 INFO:teuthology.orchestra.run.smithi049.stdout:3185 punch from 0x71512de to 0x715f9e4, (0xe706 bytes) 2024-04-24T18:08:23.598 INFO:teuthology.orchestra.run.smithi049.stdout:3187 punch from 0x493ce42 to 0x49487ca, (0xb988 bytes) 2024-04-24T18:08:23.607 INFO:teuthology.orchestra.run.smithi049.stdout:3188 trunc from 0xd70a580 to 0x4c09ade 2024-04-24T18:08:23.614 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:23.622+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:23.617 INFO:teuthology.orchestra.run.smithi049.stdout:3190 read 0x3ae879e thru 0x3af24ac (0x9d0f bytes) 2024-04-24T18:08:23.618 INFO:teuthology.orchestra.run.smithi049.stdout:3191 read 0x4bb86eb thru 0x4bbca2e (0x4344 bytes) 2024-04-24T18:08:23.618 INFO:teuthology.orchestra.run.smithi049.stdout:3192 read 0x26444e0 thru 0x264dd94 (0x98b5 bytes) 2024-04-24T18:08:23.618 INFO:teuthology.orchestra.run.smithi049.stdout:3193 read 0x2d47c8b thru 0x2d56172 (0xe4e8 bytes) 2024-04-24T18:08:23.619 INFO:teuthology.orchestra.run.smithi049.stdout:3194 trunc from 0x4c09ade to 0x571bd87 2024-04-24T18:08:23.623 INFO:teuthology.orchestra.run.smithi049.stdout:3195 punch from 0x1b921de to 0x1ba17aa, (0xf5cc bytes) 2024-04-24T18:08:23.632 INFO:teuthology.orchestra.run.smithi049.stdout:3196 writesame 0x916701f thru 0x917001d data_size 0x1(0x8fff bytes) 2024-04-24T18:08:23.633 INFO:teuthology.orchestra.run.smithi049.stdout:3197 read 0x5ebfae2 thru 0x5ecf0b3 (0xf5d2 bytes) 2024-04-24T18:08:23.633 INFO:teuthology.orchestra.run.smithi049.stdout:3199 punch from 0x73087f7 to 0x7312882, (0xa08b bytes) 2024-04-24T18:08:23.633 INFO:teuthology.orchestra.run.smithi049.stdout:3200 read 0x87f342 thru 0x88417f (0x4e3e bytes) 2024-04-24T18:08:23.633 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:23.633 INFO:teuthology.orchestra.run.smithi049.stdout:3203 writesame 0x7580663 thru 0x7583e40 data_size 0x1(0x37de bytes) 2024-04-24T18:08:23.639 INFO:teuthology.orchestra.run.smithi049.stdout:3204 write 0x5812927 thru 0x581fafe (0xd1d8 bytes) 2024-04-24T18:08:23.645 INFO:teuthology.orchestra.run.smithi049.stdout:3205 trunc from 0x917001e to 0xaf4b6be 2024-04-24T18:08:23.650 INFO:teuthology.orchestra.run.smithi049.stdout:3208 read 0x949205d thru 0x9499497 (0x743b bytes) 2024-04-24T18:08:23.651 INFO:teuthology.orchestra.run.smithi049.stdout:3209 writesame 0x8c6d3a8 thru 0x8c71140 data_size 0x1(0x3d99 bytes) 2024-04-24T18:08:23.666 INFO:teuthology.orchestra.run.smithi049.stdout:3211 write 0xc7c19fd thru 0xc7c2cea (0x12ee bytes) 2024-04-24T18:08:23.669 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:23.669 INFO:teuthology.orchestra.run.smithi049.stdout:3213 writesame 0x18bf2dd thru 0x18c9722 data_size 0x6(0xa446 bytes) 2024-04-24T18:08:23.673 INFO:teuthology.orchestra.run.smithi049.stdout:3216 write 0xba984fb thru 0xbaa64e4 (0xdfea bytes) 2024-04-24T18:08:23.677 INFO:teuthology.orchestra.run.smithi049.stdout:3218 clone 28 order 20 su 8192 sc 3 2024-04-24T18:08:23.702 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:23.702 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:23.712 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:23.748 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:08:23.754 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:23.761+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:23.816 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:23.824+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:23.843 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone27 from 0xc7c2ceb (overlap 0x12d443d) to 0x0 2024-04-24T18:08:23.903 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:23.903 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:23.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:23.917 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:23.924+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:23.987 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #26, image image_client.0-clone26 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent27 2024-04-24T18:08:24.018 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:24.026+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:24.020 INFO:teuthology.orchestra.run.smithi049.stdout:3220 trunc from 0xc7c2ceb to 0x7442bfa 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:4.e 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:4.a 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:4.5 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:4.1f 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:4.13 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:4.12 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:1.7 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:4.18 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:1.6 2024-04-24T18:08:24.036 INFO:teuthology.orchestra.run.smithi078.stdout:1.3 2024-04-24T18:08:24.042 INFO:teuthology.orchestra.run.smithi049.stdout:3221 read 0x5eff835 thru 0x5f0d3e1 (0xdbad bytes) 2024-04-24T18:08:24.044 INFO:teuthology.orchestra.run.smithi049.stdout:3223 trunc from 0x7442bfa to 0x87f9f36 2024-04-24T18:08:24.053 INFO:teuthology.orchestra.run.smithi049.stdout:3224 trunc from 0x87f9f36 to 0x681fdc7 2024-04-24T18:08:24.068 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:24.068 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:24.081 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:24.085 INFO:teuthology.orchestra.run.smithi049.stdout:3225 write 0xaefb57e thru 0xaf07617 (0xc09a bytes) 2024-04-24T18:08:24.093 INFO:teuthology.orchestra.run.smithi049.stdout:3226 read 0x74af1d7 thru 0x74bd75c (0xe586 bytes) 2024-04-24T18:08:24.093 INFO:teuthology.orchestra.run.smithi049.stdout:3227 read 0x7b7c8f1 thru 0x7b867cd (0x9edd bytes) 2024-04-24T18:08:24.093 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.093 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.093 INFO:teuthology.orchestra.run.smithi049.stdout:3230 writesame 0x4f1b23a thru 0x4f1c6fb data_size 0x1(0x14c2 bytes) 2024-04-24T18:08:24.103 INFO:teuthology.orchestra.run.smithi049.stdout:3233 write 0x70d06d9 thru 0x70e0041 (0xf969 bytes) 2024-04-24T18:08:24.109 INFO:teuthology.orchestra.run.smithi049.stdout:3234 read 0x2631af2 thru 0x263422d (0x273c bytes) 2024-04-24T18:08:24.110 INFO:teuthology.orchestra.run.smithi049.stdout:3235 punch from 0xb7a37b to 0xb86e18, (0xca9d bytes) 2024-04-24T18:08:24.110 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.118 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:24.126+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:24.121 INFO:teuthology.orchestra.run.smithi049.stdout:3237 writesame 0xe3df312 thru 0xe3e0fe2 data_size 0x1(0x1cd1 bytes) 2024-04-24T18:08:24.121 INFO:teuthology.orchestra.run.smithi049.stdout:3238 punch from 0xb1d0a63 to 0xb1df690, (0xec2d bytes) 2024-04-24T18:08:24.121 INFO:teuthology.orchestra.run.smithi049.stdout:truncating to largest ever: 0xee38272 2024-04-24T18:08:24.121 INFO:teuthology.orchestra.run.smithi049.stdout:3241 trunc from 0xe3e0fe3 to 0xee38272 2024-04-24T18:08:24.127 INFO:teuthology.orchestra.run.smithi049.stdout:3244 read 0x651a407 thru 0x6528d60 (0xe95a bytes) 2024-04-24T18:08:24.127 INFO:teuthology.orchestra.run.smithi049.stdout:3245 writesame 0x34a3a0e thru 0x34b322d data_size 0x50(0xf820 bytes) 2024-04-24T18:08:24.140 INFO:teuthology.orchestra.run.smithi049.stdout:3246 writesame 0xba59699 thru 0xba61094 data_size 0x2(0x79fc bytes) 2024-04-24T18:08:24.151 INFO:teuthology.orchestra.run.smithi049.stdout:3247 write 0xe48fa04 thru 0xe49021e (0x81b bytes) 2024-04-24T18:08:24.156 INFO:teuthology.orchestra.run.smithi049.stdout:3249 punch from 0x8b16c3a to 0x8b20ea1, (0xa267 bytes) 2024-04-24T18:08:24.156 INFO:teuthology.orchestra.run.smithi049.stdout:3250 write 0x31c4259 thru 0x31ca503 (0x62ab bytes) 2024-04-24T18:08:24.170 INFO:teuthology.orchestra.run.smithi049.stdout:3251 write 0x6b432b0 thru 0x6b4b1b3 (0x7f04 bytes) 2024-04-24T18:08:24.175 INFO:teuthology.orchestra.run.smithi049.stdout:3253 read 0x49bb9c9 thru 0x49c47d1 (0x8e09 bytes) 2024-04-24T18:08:24.175 INFO:teuthology.orchestra.run.smithi049.stdout:3254 punch from 0xb56c812 to 0xb57634a, (0x9b38 bytes) 2024-04-24T18:08:24.175 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.175 INFO:teuthology.orchestra.run.smithi049.stdout:3258 write 0x25ec0d thru 0x265e21 (0x7215 bytes) 2024-04-24T18:08:24.208 INFO:teuthology.orchestra.run.smithi049.stdout:3260 trunc from 0xee38272 to 0x6e0650c 2024-04-24T18:08:24.219 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:24.227+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:24.221 INFO:teuthology.orchestra.run.smithi049.stdout:3261 trunc from 0x6e0650c to 0x80a8544 2024-04-24T18:08:24.226 INFO:teuthology.orchestra.run.smithi049.stdout:3262 read 0x11a4ede thru 0x11a9d80 (0x4ea3 bytes) 2024-04-24T18:08:24.226 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.226 INFO:teuthology.orchestra.run.smithi049.stdout:3264 write 0x5783030 thru 0x578713d (0x410e bytes) 2024-04-24T18:08:24.240 INFO:teuthology.orchestra.run.smithi049.stdout:3265 read 0x52179b1 thru 0x5220bcf (0x921f bytes) 2024-04-24T18:08:24.247 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:24.247 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:24.252 INFO:teuthology.orchestra.run.smithi049.stdout:3266 write 0xd4424ed thru 0xd44ceb4 (0xa9c8 bytes) 2024-04-24T18:08:24.257 INFO:teuthology.orchestra.run.smithi049.stdout:3267 punch from 0x3827a60 to 0x382a1f1, (0x2791 bytes) 2024-04-24T18:08:24.257 INFO:teuthology.orchestra.run.smithi049.stdout:3268 write 0x30ec38a thru 0x30f7f2a (0xbba1 bytes) 2024-04-24T18:08:24.258 INFO:teuthology.orchestra.run.smithi049.stdout:3269 writesame 0xb68251f thru 0xb686027 data_size 0x1(0x3b09 bytes) 2024-04-24T18:08:24.259 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:24.266 INFO:teuthology.orchestra.run.smithi049.stdout:3270 trunc from 0xd44ceb5 to 0xd2f629e 2024-04-24T18:08:24.273 INFO:teuthology.orchestra.run.smithi049.stdout:3271 read 0xa5f6d85 thru 0xa604422 (0xd69e bytes) 2024-04-24T18:08:24.273 INFO:teuthology.orchestra.run.smithi049.stdout:3272 punch from 0x501787f to 0x501f8b9, (0x803a bytes) 2024-04-24T18:08:24.273 INFO:teuthology.orchestra.run.smithi049.stdout:3273 read 0x5f8c475 thru 0x5f98038 (0xbbc4 bytes) 2024-04-24T18:08:24.273 INFO:teuthology.orchestra.run.smithi049.stdout:3274 writesame 0xb88e120 thru 0xb89b91d data_size 0x1(0xd7fe bytes) 2024-04-24T18:08:24.292 INFO:teuthology.orchestra.run.smithi049.stdout:3275 read 0x6896290 thru 0x6897ebf (0x1c30 bytes) 2024-04-24T18:08:24.292 INFO:teuthology.orchestra.run.smithi049.stdout:3276 writesame 0xa45e91b thru 0xa46a8b7 data_size 0x53(0xbf9d bytes) 2024-04-24T18:08:24.296 INFO:teuthology.orchestra.run.smithi049.stdout:3277 write 0xa08da6e thru 0xa0911fc (0x378f bytes) 2024-04-24T18:08:24.301 INFO:teuthology.orchestra.run.smithi049.stdout:3278 write 0x8cefe4c thru 0x8cf60cc (0x6281 bytes) 2024-04-24T18:08:24.306 INFO:teuthology.orchestra.run.smithi049.stdout:3279 read 0x39bc47 thru 0x39ef91 (0x334b bytes) 2024-04-24T18:08:24.319 INFO:teuthology.orchestra.run.smithi049.stdout:3280 trunc from 0xd2f629e to 0x26acf24 2024-04-24T18:08:24.320 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:24.328+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:24.359 INFO:teuthology.orchestra.run.smithi049.stdout:3281 writesame 0x37a90f1 thru 0x37b393e data_size 0x2b(0xa84e bytes) 2024-04-24T18:08:24.379 INFO:teuthology.orchestra.run.smithi049.stdout:3282 write 0xebf0ec9 thru 0xebf9b19 (0x8c51 bytes) 2024-04-24T18:08:24.388 INFO:teuthology.orchestra.run.smithi049.stdout:3283 trunc from 0xebf9b1a to 0xaea6ad0 2024-04-24T18:08:24.398 INFO:teuthology.orchestra.run.smithi049.stdout:3284 read 0x49d9a77 thru 0x49dcbb2 (0x313c bytes) 2024-04-24T18:08:24.399 INFO:teuthology.orchestra.run.smithi049.stdout:3285 writesame 0x4873a35 thru 0x48756c7 data_size 0x23(0x1c93 bytes) 2024-04-24T18:08:24.406 INFO:teuthology.orchestra.run.smithi049.stdout:3286 trunc from 0xaea6ad0 to 0x748ccc3 2024-04-24T18:08:24.412 INFO:teuthology.orchestra.run.smithi049.stdout:3288 write 0x556b56d thru 0x557b242 (0xfcd6 bytes) 2024-04-24T18:08:24.416 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.416 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.416 INFO:teuthology.orchestra.run.smithi049.stdout:3291 read 0x47c5040 thru 0x47d1952 (0xc913 bytes) 2024-04-24T18:08:24.416 INFO:teuthology.orchestra.run.smithi049.stdout:3292 punch from 0xb4871e to 0xb57582, (0xee64 bytes) 2024-04-24T18:08:24.417 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.417 INFO:teuthology.orchestra.run.smithi049.stdout:3294 read 0x27511cd thru 0x275cad8 (0xb90c bytes) 2024-04-24T18:08:24.421 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:24.429+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:24.424 INFO:teuthology.orchestra.run.smithi049.stdout:3295 write 0x9110a9f thru 0x911668b (0x5bed bytes) 2024-04-24T18:08:24.425 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:24.425 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:24.429 INFO:teuthology.orchestra.run.smithi049.stdout:3297 punch from 0x4653111 to 0x465ca16, (0x9905 bytes) 2024-04-24T18:08:24.429 INFO:teuthology.orchestra.run.smithi049.stdout:3298 punch from 0x1b28eb4 to 0x1b2effe, (0x614a bytes) 2024-04-24T18:08:24.429 INFO:teuthology.orchestra.run.smithi049.stdout:3299 trunc from 0x911668c to 0x8cd064e 2024-04-24T18:08:24.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:24.449 INFO:teuthology.orchestra.run.smithi049.stdout:3300 write 0xc4493d1 thru 0xc44db49 (0x4779 bytes) 2024-04-24T18:08:24.453 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.454 INFO:teuthology.orchestra.run.smithi049.stdout:3302 writesame 0xa9906ba thru 0xa99ae07 data_size 0x5(0xa74e bytes) 2024-04-24T18:08:24.461 INFO:teuthology.orchestra.run.smithi049.stdout:3303 write 0xb07f290 thru 0xb0858bc (0x662d bytes) 2024-04-24T18:08:24.467 INFO:teuthology.orchestra.run.smithi049.stdout:3304 read 0x25a0044 thru 0x25aee39 (0xedf6 bytes) 2024-04-24T18:08:24.467 INFO:teuthology.orchestra.run.smithi049.stdout:3305 read 0x33cc9d6 thru 0x33d953d (0xcb68 bytes) 2024-04-24T18:08:24.468 INFO:teuthology.orchestra.run.smithi049.stdout:3308 trunc from 0xc44db4a to 0xd62d01d 2024-04-24T18:08:24.474 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.479 INFO:teuthology.orchestra.run.smithi049.stdout:3311 write 0xd75cc91 thru 0xd764d2d (0x809d bytes) 2024-04-24T18:08:24.484 INFO:teuthology.orchestra.run.smithi049.stdout:3313 write 0x7d0040d thru 0x7d0d989 (0xd57d bytes) 2024-04-24T18:08:24.492 INFO:teuthology.orchestra.run.smithi049.stdout:3314 punch from 0xce28041 to 0xce35fd0, (0xdf8f bytes) 2024-04-24T18:08:24.493 INFO:teuthology.orchestra.run.smithi049.stdout:3315 read 0x2801163 thru 0x280c63d (0xb4db bytes) 2024-04-24T18:08:24.493 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.493 INFO:teuthology.orchestra.run.smithi049.stdout:3317 write 0x8693840 thru 0x8698909 (0x50ca bytes) 2024-04-24T18:08:24.504 INFO:teuthology.orchestra.run.smithi049.stdout:3318 trunc from 0xd764d2e to 0x90f385c 2024-04-24T18:08:24.516 INFO:teuthology.orchestra.run.smithi049.stdout:3319 read 0x2a20121 thru 0x2a2c98e (0xc86e bytes) 2024-04-24T18:08:24.516 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.516 INFO:teuthology.orchestra.run.smithi049.stdout:3321 write 0x2670700 thru 0x267e788 (0xe089 bytes) 2024-04-24T18:08:24.517 INFO:teuthology.orchestra.run.smithi049.stdout:3324 writesame 0x48325f3 thru 0x4834dac data_size 0x2d(0x27ba bytes) 2024-04-24T18:08:24.520 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.520 INFO:teuthology.orchestra.run.smithi049.stdout:3327 read 0x34d876a thru 0x34e4ae4 (0xc37b bytes) 2024-04-24T18:08:24.520 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.520 INFO:teuthology.orchestra.run.smithi049.stdout:3329 write 0x360e70f thru 0x361bc10 (0xd502 bytes) 2024-04-24T18:08:24.521 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:24.529+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:24.524 INFO:teuthology.orchestra.run.smithi049.stdout:3330 punch from 0x30102b7 to 0x30141bf, (0x3f08 bytes) 2024-04-24T18:08:24.524 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:24.524 INFO:teuthology.orchestra.run.smithi049.stdout:3332 read 0x1bfe0ae thru 0x1bfe77e (0x6d1 bytes) 2024-04-24T18:08:24.536 INFO:teuthology.orchestra.run.smithi049.stdout:3335 writesame 0xd786934 thru 0xd78ba0f data_size 0xc(0x50dc bytes) 2024-04-24T18:08:24.536 INFO:teuthology.orchestra.run.smithi049.stdout:3338 read 0x30431ab thru 0x3047f2c (0x4d82 bytes) 2024-04-24T18:08:24.536 INFO:teuthology.orchestra.run.smithi049.stdout:3339 read 0x40c20f3 thru 0x40ccbfb (0xab09 bytes) 2024-04-24T18:08:24.536 INFO:teuthology.orchestra.run.smithi049.stdout:3340 write 0x165722 thru 0x1716e9 (0xbfc8 bytes) 2024-04-24T18:08:24.538 INFO:teuthology.orchestra.run.smithi049.stdout:3341 writesame 0x369c921 thru 0x36a3422 data_size 0x1(0x6b02 bytes) 2024-04-24T18:08:24.552 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op export --pgid 1.6 --file /var/log/ceph/exp.1.6.6 2024-04-24T18:08:24.554 INFO:teuthology.orchestra.run.smithi049.stdout:3342 trunc from 0xd78ba10 to 0x424ee58 2024-04-24T18:08:24.567 INFO:teuthology.orchestra.run.smithi049.stdout:3343 punch from 0x2c1e220 to 0x2c2114c, (0x2f2c bytes) 2024-04-24T18:08:24.567 INFO:teuthology.orchestra.run.smithi049.stdout:3345 write 0x39f6725 thru 0x39fcb52 (0x642e bytes) 2024-04-24T18:08:24.572 INFO:teuthology.orchestra.run.smithi049.stdout:3346 write 0x1a4492a thru 0x1a45647 (0xd1e bytes) 2024-04-24T18:08:24.585 INFO:teuthology.orchestra.run.smithi049.stdout:3347 trunc from 0x424ee58 to 0x7b170e2 2024-04-24T18:08:24.594 INFO:teuthology.orchestra.run.smithi049.stdout:3348 trunc from 0x7b170e2 to 0xb6d678a 2024-04-24T18:08:24.599 INFO:teuthology.orchestra.run.smithi049.stdout:3349 read 0x108ca53 thru 0x109c47e (0xfa2c bytes) 2024-04-24T18:08:24.601 INFO:teuthology.orchestra.run.smithi049.stdout:3350 read 0xe0a803 thru 0xe137db (0x8fd9 bytes) 2024-04-24T18:08:24.602 INFO:teuthology.orchestra.run.smithi049.stdout:3351 clone 29 order 18 su 2048 sc 6 2024-04-24T18:08:24.603 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:24.603 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:24.615 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:24.637 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:24.645+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:24.680 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:24.746 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:08:24.746 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:24.754+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:24.797 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:24.797 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:24.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:24.850 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:24.858+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:24.851 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone28 from 0xb6d678a (overlap 0x26acf24) to 0x1734915 2024-04-24T18:08:24.879 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:24.923 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:24.925 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:24.932+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:24.952 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:24.972 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:24.991 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:25.012 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:25.025 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895299059999999, 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:08:25.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895284520000001, 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.031 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:25.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:08:25.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895269668999999, 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.036 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:08:25.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.040 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.89525452, 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:08:25.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:08:25.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.045 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895239734, 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.046 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:25.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:08:25.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895225145, 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.051 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:08:25.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.055 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895202319999999, 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.060 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895186666000001, 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:25.065 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895171544, 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:08:25.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:08:25.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:25.070 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895155728000001, 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:25.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:08:25.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895140725999999, 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.075 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:25.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.079 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895125862, 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.080 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:08:25.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895104744999999, 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:08:25.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895089515, 2024-04-24T18:08:25.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:08:25.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.895074778, 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:08:25.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:08:25.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.889973568, 2024-04-24T18:08:25.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:25.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:08:25.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:08:25.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:08:25.102 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.889937721999999, 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.103 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:08:25.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.889917714999999, 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:08:25.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.888096524, 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.113 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 12.888071244000001, 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.118 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:08:25.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.122 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:25.123 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:25.033+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:25.123 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:25.124 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:25.124 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:25.125 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:25.125 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:25.126 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:25.127 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #27, image image_client.0-clone27 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent28 2024-04-24T18:08:25.127 INFO:teuthology.orchestra.run.smithi049.stdout:3352 punch from 0x9afe38 to 0x9b6811, (0x69d9 bytes) 2024-04-24T18:08:25.127 INFO:teuthology.orchestra.run.smithi049.stdout:3353 read 0x827a8b9 thru 0x827ee2d (0x4575 bytes) 2024-04-24T18:08:25.128 INFO:teuthology.orchestra.run.smithi049.stdout:3354 write 0x5c028fa thru 0x5c077b1 (0x4eb8 bytes) 2024-04-24T18:08:25.128 INFO:teuthology.orchestra.run.smithi049.stdout:3356 read 0x867cff1 thru 0x868aa6e (0xda7e bytes) 2024-04-24T18:08:25.128 INFO:teuthology.orchestra.run.smithi049.stdout:3357 write 0x8424cb9 thru 0x842fc37 (0xaf7f bytes) 2024-04-24T18:08:25.128 INFO:teuthology.orchestra.run.smithi049.stdout:3359 clone 30 order 20 su 16384 sc 6 2024-04-24T18:08:25.165 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:25.173+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:25.229 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:25.237+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:25.294 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:25.294 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:25.294 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:25.304 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:25.330 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:25.338+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:25.431 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:25.439+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:25.473 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:25.473 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:25.474 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:25.474 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:25.484 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:25.532 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:25.540+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:25.614 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:25.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.442945481000001, 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:25.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.423306354999999, 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:25.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.423274633, 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:25.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:25.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.423251216000001, 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:25.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:25.631 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.423221142999999, 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:25.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.635 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422175996, 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:25.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:25.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422158765000001, 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:25.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:25.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422141766999999, 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:25.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:25.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422123085999999, 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:25.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:25.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422105618, 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:25.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422089494, 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.659 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422058388, 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.660 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.663 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422039186999999, 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.664 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:25.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:25.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.422018144999999, 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.668 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:25.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:25.671 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.421263197, 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:25.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.421236694999999, 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.676 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.421219165, 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.683 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.419756005, 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:25.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.407917357000001, 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 13.407901014, 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:25.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:25.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:25.695 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:Exporting 1.6 info 1.6( empty local-lis/les=15/16 n=0 ec=15/15 lis/c=15/15 les/c/f=16/16/0 sis=15) 2024-04-24T18:08:25.697 INFO:teuthology.orchestra.run.smithi078.stderr:Export successful 2024-04-24T18:08:25.697 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:25.698 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:25.704+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:25.797 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:25.806+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:25.838 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:25.838 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:25.838 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:25.838 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:25.849 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:25.851 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone29 from 0xb6d678a (overlap 0xb6d678a) to 0x0 2024-04-24T18:08:25.933 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:25.941+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:26.000 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:26.007+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:26.047 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:26.047 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:26.047 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:26.047 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:26.048 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:26.059 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:26.103 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:26.108+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:26.202 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:26.209+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:26.203 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --force --op remove --pgid 1.6 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11780 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2666496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.959806979, 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042348956, 2024-04-24T18:08:26.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11780 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335768+0000", 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1910000000000003e-06 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335774+0000", 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9089999999999997e-06 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335778+0000", 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9339999999999999e-06 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378028+0000", 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042250303000000003 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378072+0000", 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3909000000000003e-05 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378110+0000", 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8050000000000003e-05 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11792 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:26.294 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.957794182999997, 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042313245999999999, 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11792 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337773+0000", 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337781+0000", 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3089999999999998e-06 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337787+0000", 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.285e-06 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337791+0000", 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.065e-06 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.379933+0000", 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042141913000000003 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380016+0000", 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2959000000000003e-05 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380087+0000", 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1038000000000007e-05 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.298 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11799 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2856960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.956556835000001, 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042339663999999999, 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11799 2024-04-24T18:08:26.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339013+0000", 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339019+0000", 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339023+0000", 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7689999999999998e-06 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381202+0000", 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042179054000000001 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381285+0000", 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3299000000000003e-05 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381351+0000", 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5795000000000003e-05 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.302 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.956541055000002, 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.303 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:08:26.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:08:26.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.956525149000001, 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.307 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:26.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:08:26.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:26.311 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.956473182000003, 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.315 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11805 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2920448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.955566482000002, 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042290415999999997, 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11805 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340003+0000", 2024-04-24T18:08:26.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.514e-06 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340010+0000", 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340014+0000", 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.805e-06 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382196+0000", 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042182175000000002 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382236+0000", 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0265999999999997e-05 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382292+0000", 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5791000000000002e-05 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.319 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.955549707999999, 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.320 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:08:26.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.955533076000002, 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.324 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:26.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.328 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11808 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2957312~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340690+0000", 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.954877875999998, 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042195728000000002, 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11808 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340690+0000", 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340690+0000", 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340052+0000", 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999362 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340695+0000", 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00064299599999999995 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340705+0000", 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0369e-05 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340711+0000", 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0460000000000002e-06 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382725+0000", 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042013834999999999 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382812+0000", 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7158999999999997e-05 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382886+0000", 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3274000000000004e-05 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.332 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11809 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2965504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.954849623000001, 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042364339000000001, 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11809 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.333 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:26.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340721+0000", 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1370000000000002e-06 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340727+0000", 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.533e-06 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340732+0000", 2024-04-24T18:08:26.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5329999999999996e-06 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382924+0000", 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042192039000000001 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383012+0000", 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7669000000000004e-05 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383083+0000", 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0988999999999994e-05 2024-04-24T18:08:26.336 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.954831472999999, 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.337 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:08:26.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:08:26.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.954814970999998, 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.341 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11814 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3012608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.953920388999997, 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042354992000000001, 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.345 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11814 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341648+0000", 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.372e-06 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341652+0000", 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0690000000000002e-06 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341654+0000", 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0559999999999999e-06 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383852+0000", 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042198125000000003 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383934+0000", 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2021999999999995e-05 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384002+0000", 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8669000000000003e-05 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.953912105000001, 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.349 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:26.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:08:26.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:08:26.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.353 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.953904080000001, 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:26.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:08:26.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:08:26.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11819 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3065856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.952872370999998, 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042208806000000001, 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11819 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342698+0000", 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6879999999999999e-06 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342704+0000", 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0090000000000004e-06 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342708+0000", 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9029999999999997e-06 2024-04-24T18:08:26.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384745+0000", 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042037243000000002 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384833+0000", 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7769999999999995e-05 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384904+0000", 2024-04-24T18:08:26.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1632999999999997e-05 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11820 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3076096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.952855706999998, 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042328461999999997, 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.362 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11820 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342714+0000", 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342720+0000", 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8479999999999997e-06 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342724+0000", 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6430000000000001e-06 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384944+0000", 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042220698000000001 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384992+0000", 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7441999999999997e-05 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385041+0000", 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8715999999999999e-05 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.952839937999997, 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.366 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 36.952823969000001, 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.370 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:26.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:08:26.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:08:26.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.374 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.375 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:26.376 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:26.377 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:26.383+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:26.477 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:26.485+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:26.557 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95865 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 2523136~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.557 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564054+0000", 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00046319900000000002, 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000502657, 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95865 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564054+0000", 2024-04-24T18:08:26.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564054+0000", 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564053+0000", 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564057+0000", 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2740000000000001e-06 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564065+0000", 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4739999999999999e-06 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564068+0000", 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564473+0000", 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00040456900000000002 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564533+0000", 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9719000000000002e-05 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95866 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564073+0000", 2024-04-24T18:08:26.561 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00044369500000000003, 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00058298200000000001, 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95866 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564073+0000", 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564073+0000", 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564073+0000", 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564075+0000", 2024-04-24T18:08:26.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2740000000000002e-06 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564081+0000", 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6520000000000003e-06 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564084+0000", 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5300000000000001e-06 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95881 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564088+0000", 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00042829899999999999, 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00063323100000000005, 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95881 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564088+0000", 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564088+0000", 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564088+0000", 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564090+0000", 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2979999999999999e-06 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564096+0000", 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.049e-06 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564100+0000", 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.709e-06 2024-04-24T18:08:26.567 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95882 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564113+0000", 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00040357499999999998, 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00067021500000000005, 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95882 2024-04-24T18:08:26.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564113+0000", 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564113+0000", 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564113+0000", 2024-04-24T18:08:26.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564115+0000", 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564121+0000", 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0499999999999997e-06 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564125+0000", 2024-04-24T18:08:26.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0339999999999998e-06 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95897 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564130+0000", 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00038699600000000002, 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00071524100000000003, 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95897 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564130+0000", 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564130+0000", 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564129+0000", 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564131+0000", 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0700000000000001e-06 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564138+0000", 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3090000000000001e-06 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564141+0000", 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.681e-06 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95898 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564146+0000", 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00037119699999999998, 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00075926800000000001, 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.574 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95898 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564146+0000", 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564146+0000", 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564145+0000", 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564148+0000", 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5210000000000001e-06 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564154+0000", 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8690000000000001e-06 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564157+0000", 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5379999999999998e-06 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95915 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564161+0000", 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00035541899999999999, 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00080262599999999997, 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.577 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95915 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564161+0000", 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564161+0000", 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564161+0000", 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564163+0000", 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2369999999999999e-06 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564169+0000", 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4519999999999996e-06 2024-04-24T18:08:26.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564172+0000", 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7409999999999998e-06 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95932 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564176+0000", 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00034028200000000002, 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00084764300000000003, 2024-04-24T18:08:26.580 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95932 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564176+0000", 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564176+0000", 2024-04-24T18:08:26.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564176+0000", 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564188+0000", 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1712999999999999e-05 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564194+0000", 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4019999999999999e-06 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564198+0000", 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8469999999999998e-06 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95948 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564203+0000", 2024-04-24T18:08:26.583 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00031387899999999998, 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00088111899999999998, 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95948 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564203+0000", 2024-04-24T18:08:26.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564203+0000", 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564202+0000", 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564205+0000", 2024-04-24T18:08:26.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2239999999999998e-06 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564210+0000", 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7960000000000001e-06 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564214+0000", 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9060000000000004e-06 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95964 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564220+0000", 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00029714699999999999, 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00092617199999999995, 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95964 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564220+0000", 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564220+0000", 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564219+0000", 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564221+0000", 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2350000000000002e-06 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564227+0000", 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4060000000000004e-06 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564231+0000", 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9469999999999996e-06 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.589 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95980 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564235+0000", 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00028162700000000001, 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00096946800000000002, 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95980 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.590 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564235+0000", 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564235+0000", 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564235+0000", 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564237+0000", 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3800000000000001e-06 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564243+0000", 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1240000000000002e-06 2024-04-24T18:08:26.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564247+0000", 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7560000000000001e-06 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:95995 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564418+0000", 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 9.8808999999999995e-05, 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00084635299999999999, 2024-04-24T18:08:26.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 95995 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564418+0000", 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564418+0000", 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564417+0000", 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564422+0000", 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3560000000000003e-06 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564430+0000", 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1629999999999996e-06 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564435+0000", 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8919999999999999e-06 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:96011 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.596 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564440+0000", 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 7.6655000000000004e-05, 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00090259900000000002, 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 96011 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564440+0000", 2024-04-24T18:08:26.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564440+0000", 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564440+0000", 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564442+0000", 2024-04-24T18:08:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.711e-06 2024-04-24T18:08:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564448+0000", 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3460000000000001e-06 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564451+0000", 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6710000000000001e-06 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:96027 4.1f 4.b28ec85f (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.564470+0000", 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.6309e-05, 2024-04-24T18:08:26.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093737099999999999, 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 96027 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564470+0000", 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564470+0000", 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564470+0000", 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564473+0000", 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7070000000000001e-06 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564479+0000", 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1850000000000001e-06 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.564483+0000", 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0570000000000003e-06 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.603 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-24T18:08:26.604 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 14 2024-04-24T18:08:26.604 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:26.605 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:26.607 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:26.615+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:26.707 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:26.714+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:26.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:08:26.793 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:26.793 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:26.793 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:26.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:26.807 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:26.815+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:26.908 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:26.916+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:26.971 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:26.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92803 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 856064~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379022+0000", 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59916254499999999, 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048635280000000003, 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92803 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.972 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379022+0000", 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379022+0000", 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379022+0000", 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379024+0000", 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1490000000000001e-06 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379030+0000", 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5769999999999999e-06 2024-04-24T18:08:26.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379034+0000", 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.489e-06 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427486+0000", 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048452637999999999 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427566+0000", 2024-04-24T18:08:26.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9179999999999997e-05 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427657+0000", 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.1556000000000002e-05 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92804 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 864256~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379038+0000", 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59914621599999995, 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048818509000000003, 2024-04-24T18:08:26.976 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92804 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379038+0000", 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379038+0000", 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379038+0000", 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379043+0000", 2024-04-24T18:08:26.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.101e-06 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379049+0000", 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0360000000000003e-06 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379053+0000", 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0180000000000003e-06 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427684+0000", 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048631504999999998 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427766+0000", 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1811000000000007e-05 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427857+0000", 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0467e-05 2024-04-24T18:08:26.980 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92805 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 872448~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379058+0000", 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59912689500000005, 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048997129, 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.981 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92805 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379058+0000", 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379058+0000", 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379057+0000", 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379060+0000", 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5359999999999999e-06 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379065+0000", 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8050000000000003e-06 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379069+0000", 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8639999999999998e-06 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427885+0000", 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048815363000000001 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.427966+0000", 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0939000000000006e-05 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428055+0000", 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9133000000000002e-05 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92806 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 880640~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379080+0000", 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59910432999999996, 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049166077000000002, 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92806 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.986 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379080+0000", 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379080+0000", 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379080+0000", 2024-04-24T18:08:26.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379082+0000", 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3769999999999999e-06 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379091+0000", 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7150000000000004e-06 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379095+0000", 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2699999999999998e-06 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428084+0000", 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048988455 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428162+0000", 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8597999999999995e-05 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428246+0000", 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3999000000000007e-05 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92807 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 888832~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379100+0000", 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59908491200000002, 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049346854000000002, 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.990 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92807 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379100+0000", 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379100+0000", 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379099+0000", 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379113+0000", 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4045e-05 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379125+0000", 2024-04-24T18:08:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1806e-05 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379130+0000", 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1569999999999999e-06 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428273+0000", 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049143011 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428354+0000", 2024-04-24T18:08:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1007000000000006e-05 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428446+0000", 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2232999999999996e-05 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92808 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 897024~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379143+0000", 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59904170999999995, 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049453761999999998, 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92808 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379143+0000", 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379143+0000", 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379142+0000", 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379145+0000", 2024-04-24T18:08:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7700000000000002e-06 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379156+0000", 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.078e-05 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379160+0000", 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2950000000000004e-06 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428474+0000", 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.04931381 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428540+0000", 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6439999999999994e-05 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428597+0000", 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6107000000000003e-05 2024-04-24T18:08:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92809 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 905216~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379165+0000", 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59901974000000002, 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049600478000000003, 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92809 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379165+0000", 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379165+0000", 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379164+0000", 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379167+0000", 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.48e-06 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379179+0000", 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2454e-05 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379184+0000", 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1489999999999996e-06 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428631+0000", 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049447647999999997 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428709+0000", 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7390000000000003e-05 2024-04-24T18:08:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428765+0000", 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6657999999999999e-05 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92810 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 913408~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379188+0000", 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59899649099999996, 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049723192999999999, 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92810 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379188+0000", 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379188+0000", 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379188+0000", 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379190+0000", 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4980000000000001e-06 2024-04-24T18:08:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379197+0000", 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5629999999999997e-06 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379204+0000", 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7980000000000006e-06 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428805+0000", 2024-04-24T18:08:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049600501999999998 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428847+0000", 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2351000000000002e-05 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428911+0000", 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3827999999999999e-05 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92811 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 921600~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379209+0000", 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59897520199999998, 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049819480999999999, 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92811 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379209+0000", 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379209+0000", 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379209+0000", 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379211+0000", 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4880000000000001e-06 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379222+0000", 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0424999999999999e-05 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379226+0000", 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9580000000000001e-06 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428936+0000", 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049710066999999997 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.428976+0000", 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0581999999999999e-05 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429029+0000", 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2289e-05 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92812 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 929792~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379335+0000", 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59884899300000005, 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049807258, 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92812 2024-04-24T18:08:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379335+0000", 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379335+0000", 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379230+0000", 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9998941 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379339+0000", 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000109205 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379356+0000", 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6773e-05 2024-04-24T18:08:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379361+0000", 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.395e-06 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429053+0000", 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049692088000000002 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429093+0000", 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9903000000000001e-05 2024-04-24T18:08:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429143+0000", 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9846999999999999e-05 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92813 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 937984~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379700+0000", 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59848469900000001, 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049557203000000001, 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92813 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379700+0000", 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379700+0000", 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379366+0000", 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9996662 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379705+0000", 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00033862299999999997 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379714+0000", 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7739999999999993e-06 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379721+0000", 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2289999999999997e-06 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429167+0000", 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049446453000000001 2024-04-24T18:08:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429207+0000", 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9848999999999997e-05 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429257+0000", 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9876999999999997e-05 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92814 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 946176~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.379732+0000", 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59845258400000001, 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049638060999999997, 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92814 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379732+0000", 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379732+0000", 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379731+0000", 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379734+0000", 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8660000000000002e-06 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379740+0000", 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0750000000000003e-06 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379744+0000", 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0080000000000004e-06 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429280+0000", 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049535441 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429320+0000", 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0617000000000001e-05 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429370+0000", 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9536000000000001e-05 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92815 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 954368~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.380678+0000", 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59750610599999998, 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048804247000000002, 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92815 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380678+0000", 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380678+0000", 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.379749+0000", 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9990702 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380683+0000", 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093446899999999995 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380694+0000", 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1311999999999999e-05 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380700+0000", 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8279999999999999e-06 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429394+0000", 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048693657000000001 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429433+0000", 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9266e-05 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429483+0000", 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9459000000000002e-05 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92816 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 962560~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.380707+0000", 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59747782299999996, 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048888441999999997, 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92816 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380707+0000", 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380707+0000", 2024-04-24T18:08:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380706+0000", 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380708+0000", 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0930000000000001e-06 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380712+0000", 2024-04-24T18:08:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380714+0000", 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2989999999999999e-06 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429507+0000", 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048792462000000002 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429546+0000", 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8970999999999997e-05 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429595+0000", 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9166000000000003e-05 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92817 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 970752~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.380717+0000", 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59746744299999999, 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049000401999999998, 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92817 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380717+0000", 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380717+0000", 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380717+0000", 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380718+0000", 2024-04-24T18:08:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4959999999999999e-06 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380721+0000", 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1e-06 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380723+0000", 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0719999999999998e-06 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429626+0000", 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048902080000000001 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429665+0000", 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9511000000000002e-05 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429717+0000", 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2357e-05 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92818 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 978944~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.380726+0000", 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59745871299999997, 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049101348000000003, 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92818 2024-04-24T18:08:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380726+0000", 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380726+0000", 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380726+0000", 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380727+0000", 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.381e-06 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380730+0000", 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9699999999999999e-06 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380732+0000", 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429740+0000", 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049008399000000001 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429780+0000", 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9439e-05 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429827+0000", 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7267e-05 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92819 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 987136~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.380735+0000", 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59744994200000001, 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049202831000000002, 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92819 2024-04-24T18:08:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380735+0000", 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380735+0000", 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380734+0000", 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380741+0000", 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5699999999999998e-06 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380745+0000", 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.693e-06 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.380747+0000", 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2189999999999998e-06 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429850+0000", 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049103736000000002 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429890+0000", 2024-04-24T18:08:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9078000000000001e-05 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.429937+0000", 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7815000000000002e-05 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92823 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 1019904~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.381702+0000", 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59648278399999999, 2024-04-24T18:08:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048679851000000003, 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92823 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381702+0000", 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381702+0000", 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381701+0000", 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381703+0000", 2024-04-24T18:08:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5409999999999999e-06 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381706+0000", 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2370000000000001e-06 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381708+0000", 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9750000000000001e-06 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.430295+0000", 2024-04-24T18:08:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048586662000000003 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.430334+0000", 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9307000000000002e-05 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.430382+0000", 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7348999999999997e-05 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92824 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 1028096~2048] snapc 0=[] RETRY=4 ondisk+retry+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.381711+0000", 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59647301500000005, 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048788757000000002, 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92824 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381711+0000", 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381711+0000", 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381711+0000", 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381713+0000", 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4839999999999999e-06 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381716+0000", 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1020000000000001e-06 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.381718+0000", 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.430404+0000", 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048686172 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.430443+0000", 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9218000000000003e-05 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.430500+0000", 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6983e-05 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92894 4.8 4:1458aa2e:::rbd_data.11748c7976ea.0000000000000015:20 [read 542720~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.382733+0000", 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.59545135800000004, 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048667323999999998, 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92894 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.382733+0000", 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.382733+0000", 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.382733+0000", 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.382735+0000", 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.243e-06 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.382741+0000", 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2339999999999999e-06 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.382745+0000", 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8619999999999996e-06 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.431236+0000", 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.048490395999999998 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.431315+0000", 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9126000000000001e-05 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.431400+0000", 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5780999999999997e-05 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:27.059 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:27.060 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:27.061 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:27.069+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:27.126 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:27.172 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:27.180+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:27.254 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:27.255 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:27.255 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:27.264 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:27.271+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:27.266 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:27.345 INFO:teuthology.orchestra.run.smithi078.stdout: marking collection for removal 2024-04-24T18:08:27.345 INFO:teuthology.orchestra.run.smithi078.stdout:setting '_remove' omap key 2024-04-24T18:08:27.345 INFO:teuthology.orchestra.run.smithi078.stdout:finish_remove_pgs 1.6_head removing 1.6 2024-04-24T18:08:27.350 INFO:teuthology.orchestra.run.smithi078.stdout:Remove successful 2024-04-24T18:08:27.355 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:27.363 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:27.371+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:27.376 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:27.397 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:27.411 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:27.411 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:27.411 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:27.421 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:27.423 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:27.441 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:27.460 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:27.464 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:27.472+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:27.470 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #28, image image_client.0-clone28 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent29 2024-04-24T18:08:27.490 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:27.491 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:27.497 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:27.497 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:27.566 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:27.574+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:27.569 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:27.569 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:27.572 INFO:teuthology.orchestra.run.smithi049.stdout:3361 write 0x57d8394 thru 0x57dd002 (0x4c6f bytes) 2024-04-24T18:08:27.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:27.585 INFO:teuthology.orchestra.run.smithi049.stdout:3362 read 0x93393c7 thru 0x933fdb9 (0x69f3 bytes) 2024-04-24T18:08:27.585 INFO:teuthology.orchestra.run.smithi049.stdout:3363 write 0x932a62 thru 0x94073d (0xdcdc bytes) 2024-04-24T18:08:27.668 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:27.674+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:27.708 INFO:teuthology.orchestra.run.smithi049.stdout:3364 trunc from 0xb6d678a to 0x1c45cc0 2024-04-24T18:08:27.771 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:27.779+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:27.814 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:08:27.832 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:27.833 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:27.847 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:27.852 INFO:teuthology.orchestra.run.smithi049.stdout:3365 read 0x5510e3 thru 0x553bd1 (0x2aef bytes) 2024-04-24T18:08:27.862 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.1.6.6 2024-04-24T18:08:27.869 INFO:teuthology.orchestra.run.smithi049.stdout:3366 write 0xa275136 thru 0xa279d87 (0x4c52 bytes) 2024-04-24T18:08:27.874 INFO:teuthology.orchestra.run.smithi049.stdout:3368 read 0x88ca50a thru 0x88d1908 (0x73ff bytes) 2024-04-24T18:08:27.875 INFO:teuthology.orchestra.run.smithi049.stdout:3369 punch from 0x5455945 to 0x545a0c4, (0x477f bytes) 2024-04-24T18:08:27.875 INFO:teuthology.orchestra.run.smithi049.stdout:3370 read 0x46cc406 thru 0x46d58a2 (0x949d bytes) 2024-04-24T18:08:27.875 INFO:teuthology.orchestra.run.smithi049.stdout:3371 punch from 0x7312d7f to 0x731afb2, (0x8233 bytes) 2024-04-24T18:08:27.875 INFO:teuthology.orchestra.run.smithi049.stdout:3372 read 0x2f81c6 thru 0x302068 (0x9ea3 bytes) 2024-04-24T18:08:27.875 INFO:teuthology.orchestra.run.smithi049.stdout:3373 write 0x8b2512a thru 0x8b268c2 (0x1799 bytes) 2024-04-24T18:08:27.878 INFO:teuthology.orchestra.run.smithi049.stdout:3374 write 0x580ae0f thru 0x580be01 (0xff3 bytes) 2024-04-24T18:08:27.880 INFO:teuthology.orchestra.run.smithi049.stdout:3375 writesame 0x591e156 thru 0x5920b98 data_size 0x1(0x2a43 bytes) 2024-04-24T18:08:27.890 INFO:teuthology.orchestra.run.smithi049.stdout:3376 trunc from 0xa279d88 to 0xd1cee0d 2024-04-24T18:08:27.902 INFO:teuthology.orchestra.run.smithi049.stdout:3377 trunc from 0xd1cee0d to 0x8c5b994 2024-04-24T18:08:27.906 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:27.915+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:27.912 INFO:teuthology.orchestra.run.smithi049.stdout:3378 read 0x69ad51 thru 0x6a0ef5 (0x61a5 bytes) 2024-04-24T18:08:27.920 INFO:teuthology.orchestra.run.smithi049.stdout:3380 writesame 0x9f720f3 thru 0x9f73b3e data_size 0x3(0x1a4c bytes) 2024-04-24T18:08:27.920 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:27.920 INFO:teuthology.orchestra.run.smithi049.stdout:3382 write 0x1cceef5 thru 0x1cd9201 (0xa30d bytes) 2024-04-24T18:08:27.922 INFO:teuthology.orchestra.run.smithi049.stdout:3384 writesame 0xcb7ddf thru 0xcbc97d data_size 0x1b(0x4b9f bytes) 2024-04-24T18:08:27.969 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:27.977+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:27.989 INFO:teuthology.orchestra.run.smithi049.stdout:3386 read 0x30201b2 thru 0x3023b28 (0x3977 bytes) 2024-04-24T18:08:27.989 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:27.989 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:27.989 INFO:teuthology.orchestra.run.smithi049.stdout:3390 write 0x12ac4b6 thru 0x12baa82 (0xe5cd bytes) 2024-04-24T18:08:28.037 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:28.037 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:28.048 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:28.088 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:28.096+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:28.171 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:28.178+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:28.217 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:28.217 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:28.229 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:28.271 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:28.279+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:3392 read 0x80e8d42 thru 0x80ea223 (0x14e2 bytes) 2024-04-24T18:08:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:3393 punch from 0x55d06c5 to 0x55d096f, (0x2aa bytes) 2024-04-24T18:08:28.293 INFO:teuthology.orchestra.run.smithi049.stdout:3394 writesame 0xfc0aee thru 0xfc8456 data_size 0x1(0x7969 bytes) 2024-04-24T18:08:28.372 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:28.379+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:28.398 INFO:teuthology.orchestra.run.smithi049.stdout:3395 write 0x9fa5a70 thru 0x9fb188d (0xbe1e bytes) 2024-04-24T18:08:28.404 INFO:teuthology.orchestra.run.smithi049.stdout:3397 punch from 0x8eabce9 to 0x8eb59c5, (0x9cdc bytes) 2024-04-24T18:08:28.404 INFO:teuthology.orchestra.run.smithi049.stdout:3398 write 0x573aaa1 thru 0x573dea7 (0x3407 bytes) 2024-04-24T18:08:28.407 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:28.407 INFO:teuthology.orchestra.run.smithi049.stdout:3400 write 0x1494e11 thru 0x14a0cb6 (0xbea6 bytes) 2024-04-24T18:08:28.449 INFO:teuthology.orchestra.run.smithi049.stdout:3401 read 0x4203998 thru 0x4205ef5 (0x255e bytes) 2024-04-24T18:08:28.453 INFO:teuthology.orchestra.run.smithi049.stdout:3402 trunc from 0x9fb188e to 0xe2af461 2024-04-24T18:08:28.455 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:28.456 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:28.459 INFO:teuthology.orchestra.run.smithi049.stdout:3403 punch from 0x28785c to 0x294f0f, (0xd6b3 bytes) 2024-04-24T18:08:28.459 INFO:teuthology.orchestra.run.smithi049.stdout:3404 write 0x6f775a5 thru 0x6f7ae1f (0x387b bytes) 2024-04-24T18:08:28.462 INFO:teuthology.orchestra.run.smithi049.stdout:3405 read 0xd732069 thru 0xd741cc0 (0xfc58 bytes) 2024-04-24T18:08:28.463 INFO:teuthology.orchestra.run.smithi049.stdout:3406 read 0x5cb6dec thru 0x5cc547d (0xe692 bytes) 2024-04-24T18:08:28.463 INFO:teuthology.orchestra.run.smithi049.stdout:3407 writesame 0xd234519 thru 0xd2396e0 data_size 0x4(0x51c8 bytes) 2024-04-24T18:08:28.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:28.473 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:28.481+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:28.474 INFO:teuthology.orchestra.run.smithi049.stdout:3408 trunc from 0xe2af461 to 0x82db132 2024-04-24T18:08:28.485 INFO:teuthology.orchestra.run.smithi049.stdout:3409 writesame 0x557ae84 thru 0x5589e7e data_size 0xb(0xeffb bytes) 2024-04-24T18:08:28.490 INFO:teuthology.orchestra.run.smithi049.stdout:3411 punch from 0x24522b0 to 0x245e48c, (0xc1dc bytes) 2024-04-24T18:08:28.491 INFO:teuthology.orchestra.run.smithi049.stdout:3412 read 0x66af6de thru 0x66b433b (0x4c5e bytes) 2024-04-24T18:08:28.491 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:28.499 INFO:teuthology.orchestra.run.smithi049.stdout:3416 trunc from 0x82db132 to 0xcacca3 2024-04-24T18:08:28.515 INFO:teuthology.orchestra.run.smithi049.stdout:3417 punch from 0x844ae to 0x913bb, (0xcf0d bytes) 2024-04-24T18:08:28.515 INFO:teuthology.orchestra.run.smithi049.stdout:3418 trunc from 0xcacca3 to 0x1515a1b 2024-04-24T18:08:28.520 INFO:teuthology.orchestra.run.smithi049.stdout:3420 read 0x1ad075 thru 0x1b4be4 (0x7b70 bytes) 2024-04-24T18:08:28.521 INFO:teuthology.orchestra.run.smithi049.stdout:3421 read 0x52dc4f thru 0x530335 (0x26e7 bytes) 2024-04-24T18:08:28.522 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:28.523 INFO:teuthology.orchestra.run.smithi049.stdout:3423 clone 31 order 23 su 1048576 sc 7 2024-04-24T18:08:28.574 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:28.582+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:28.614 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:28.614 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:28.624 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:28.675 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:28.683+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:28.776 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:28.784+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:28.795 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:28.796 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:28.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:28.892 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:28.900+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:28.893 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone30 from 0x1515a1b (overlap 0xcacca3) to 0xa23d4f 2024-04-24T18:08:28.966 INFO:teuthology.orchestra.run.smithi078.stdout:get_pg_num_history pg_num_history pg_num_history(e67 pg_nums {1={15=8},2={15=1},3={18=4},4={26=32}} deleted_pools ) 2024-04-24T18:08:28.967 INFO:teuthology.orchestra.run.smithi078.stdout:Importing pgid 1.6 2024-04-24T18:08:28.967 INFO:teuthology.orchestra.run.smithi078.stdout:write_pg epoch 57 info 1.6( empty local-lis/les=15/16 n=0 ec=15/15 lis/c=15/15 les/c/f=16/16/0 sis=15) 2024-04-24T18:08:28.967 INFO:teuthology.orchestra.run.smithi078.stdout:Import successful 2024-04-24T18:08:28.977 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:28.985+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:28.994 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:28.994 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:29.005 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:29.032 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #29, image image_client.0-clone29 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent30 2024-04-24T18:08:29.048 INFO:teuthology.orchestra.run.smithi049.stdout:3424 writesame 0x6debf07 thru 0x6defe65 data_size 0x1(0x3f5f bytes) 2024-04-24T18:08:29.049 INFO:teuthology.orchestra.run.smithi049.stdout:3425 writesame 0x1ce2b9 thru 0x1db58c data_size 0x83(0xd2d4 bytes) 2024-04-24T18:08:29.071 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:29.078 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:29.086+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:29.079 INFO:teuthology.orchestra.run.smithi049.stdout:3427 write 0x8b959d3 thru 0x8b9ce7f (0x74ad bytes) 2024-04-24T18:08:29.083 INFO:teuthology.orchestra.run.smithi049.stdout:3428 read 0x2ce433c thru 0x2cf1722 (0xd3e7 bytes) 2024-04-24T18:08:29.083 INFO:teuthology.orchestra.run.smithi049.stdout:3429 read 0x41cfb91 thru 0x41d62a8 (0x6718 bytes) 2024-04-24T18:08:29.083 INFO:teuthology.orchestra.run.smithi049.stdout:3430 read 0x13fcb86 thru 0x14022d8 (0x5753 bytes) 2024-04-24T18:08:29.093 INFO:teuthology.orchestra.run.smithi049.stdout:3431 writesame 0xcf2c13b thru 0xcf3190b data_size 0x1(0x57d1 bytes) 2024-04-24T18:08:29.093 INFO:teuthology.orchestra.run.smithi049.stdout:3434 punch from 0xa06cbea to 0xa07bd47, (0xf15d bytes) 2024-04-24T18:08:29.094 INFO:teuthology.orchestra.run.smithi049.stdout:3435 write 0x7748d72 thru 0x774cb04 (0x3d93 bytes) 2024-04-24T18:08:29.096 INFO:teuthology.orchestra.run.smithi049.stdout:3436 read 0x35e7415 thru 0x35eaad3 (0x36bf bytes) 2024-04-24T18:08:29.096 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:29.096 INFO:teuthology.orchestra.run.smithi049.stdout:3438 writesame 0xc1c6541 thru 0xc1cad0c data_size 0xa(0x47cc bytes) 2024-04-24T18:08:29.099 INFO:teuthology.orchestra.run.smithi049.stdout:3439 write 0x3fa1e61 thru 0x3fa6c70 (0x4e10 bytes) 2024-04-24T18:08:29.103 INFO:teuthology.orchestra.run.smithi049.stdout:3441 punch from 0x7fc01f6 to 0x7fc0615, (0x41f bytes) 2024-04-24T18:08:29.103 INFO:teuthology.orchestra.run.smithi049.stdout:3442 write 0xa5d664e thru 0xa5deb2e (0x84e1 bytes) 2024-04-24T18:08:29.106 INFO:teuthology.orchestra.run.smithi049.stdout:3444 punch from 0x906848c to 0x906a05e, (0x1bd2 bytes) 2024-04-24T18:08:29.106 INFO:teuthology.orchestra.run.smithi049.stdout:3445 punch from 0xcbb7b7e to 0xcbc04cf, (0x8951 bytes) 2024-04-24T18:08:29.106 INFO:teuthology.orchestra.run.smithi049.stdout:3447 read 0x7f695a1 thru 0x7f74189 (0xabe9 bytes) 2024-04-24T18:08:29.106 INFO:teuthology.orchestra.run.smithi049.stdout:3448 writesame 0x5cf511c thru 0x5cf71d9 data_size 0xb(0x20be bytes) 2024-04-24T18:08:29.123 INFO:teuthology.orchestra.run.smithi049.stdout:3449 trunc from 0xcf3190c to 0xb9ab 2024-04-24T18:08:29.172 INFO:teuthology.orchestra.run.smithi049.stdout:3450 writesame 0x3237b06 thru 0x32439f6 data_size 0x1(0xbef1 bytes) 2024-04-24T18:08:29.172 INFO:teuthology.orchestra.run.smithi049.stdout:3451 punch from 0x15db34c to 0x15e7fe5, (0xcc99 bytes) 2024-04-24T18:08:29.179 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:29.186+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:3455 writesame 0x6c75469 thru 0x6c7c910 data_size 0x4(0x74a8 bytes) 2024-04-24T18:08:29.182 INFO:teuthology.orchestra.run.smithi049.stdout:3456 write 0xb0d6ef thru 0xb177c6 (0xa0d8 bytes) 2024-04-24T18:08:29.191 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:29.192 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:29.197 INFO:teuthology.orchestra.run.smithi049.stdout:3458 write 0xc2d8b53 thru 0xc2dad6f (0x221d bytes) 2024-04-24T18:08:29.201 INFO:teuthology.orchestra.run.smithi049.stdout:3459 read 0xb6dc891 thru 0xb6e017f (0x38ef bytes) 2024-04-24T18:08:29.201 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:29.201 INFO:teuthology.orchestra.run.smithi049.stdout:3461 writesame 0xad1e468 thru 0xad229d3 data_size 0x6(0x456c bytes) 2024-04-24T18:08:29.203 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:29.204 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:29.204 INFO:teuthology.orchestra.run.smithi049.stdout:3463 read 0x595507d thru 0x5958ba5 (0x3b29 bytes) 2024-04-24T18:08:29.204 INFO:teuthology.orchestra.run.smithi049.stdout:3465 read 0x1d505cf thru 0x1d52ed2 (0x2904 bytes) 2024-04-24T18:08:29.204 INFO:teuthology.orchestra.run.smithi049.stdout:3466 read 0x2e7cba7 thru 0x2e89d5f (0xd1b9 bytes) 2024-04-24T18:08:29.204 INFO:teuthology.orchestra.run.smithi049.stdout:3468 clone 32 order 22 su 65536 sc 9 2024-04-24T18:08:29.280 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:29.289+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:29.371 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:29.371 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:29.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:29.382 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:29.389+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:29.482 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:29.490+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:29.487 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f /var/log/ceph/exp.1.6.6 2024-04-24T18:08:29.498 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:29.553 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:08:29.554 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:29.554 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:29.564 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:29.605 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:29.613+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:29.648 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:29.670 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:29.684 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:29.692+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:29.686 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:29.706 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:29.725 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:29.744 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:29.745 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:29.745 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:29.746 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:29.746 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:29.746 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:29.756 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:29.764 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:29.764 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:29.765 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:29.765 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:29.765 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:29.765 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:29.765 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:29.765 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:29.765 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:29.767 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:29.768 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:29.785 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:29.793+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:29.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:08:29.886 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:29.894+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:29.911 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.911 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.739817336000002, 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:29.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:29.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.915 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.72017821, 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:29.919 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.720146488000001, 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.720123071, 2024-04-24T18:08:29.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.720092997999998, 2024-04-24T18:08:29.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.719047850999999, 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:29.931 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:29.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:29.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:29.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.719030620000002, 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.935 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:29.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:29.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.719013621999999, 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.939 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:29.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:29.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718994940999998, 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.943 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718977472999999, 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.947 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718961349000001, 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.951 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718930242999999, 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.955 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718911041999998, 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.959 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:29.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:29.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718890000000002, 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.963 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:29.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:29.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:29.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718135052000001, 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.967 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.970 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.71810855, 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.974 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.718091019999999, 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.978 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.716627859999999, 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.704789212000001, 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:29.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:29.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 17.704772868999999, 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:29.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:29.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:29.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:29.990 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:29.992 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:29.992 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone31 from 0xc2dad70 (overlap 0xb9ab) to 0x2522 2024-04-24T18:08:29.993 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:08:29.993 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":76,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:08:29.784426+0000","last_up_change":"2024-04-24T18:08:21.732141+0000","last_in_change":"2024-04-24T18:07:55.610486+0000","flags":"noscrub,nodeep-scrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5806080,"flags_set":["nodeep-scrub","noscrub","pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-24T18:07:18.850062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-24T18:07:19.353026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T18:07:25.800822+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":1,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T18:07:35.513947+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"76","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":35,"snap_epoch":76,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":69,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":45,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":43544},{"type":"v1","addr":"172.21.15.78:6825","nonce":43544}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":43544},{"type":"v1","addr":"172.21.15.78:6827","nonce":43544}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":43544},{"type":"v1","addr":"172.21.15.78:6831","nonce":43544}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":43544},{"type":"v1","addr":"172.21.15.78:6829","nonce":43544}]},"public_addr":"172.21.15.78:6825/43544","cluster_addr":"172.21.15.78:6827/43544","heartbeat_back_addr":"172.21.15.78:6831/43544","heartbeat_front_addr":"172.21.15.78:6829/43544","state":["exists"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":69},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:08:29.994 INFO:tasks.ceph.ceph_manager.ceph:[{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-24T18:07:18.850062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-04-24T18:07:19.353026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-04-24T18:07:25.800822+0000', 'flags': 9, 'flags_names': 'hashpspool,incomplete_clones', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-04-24T18:07:35.513947+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '76', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 35, 'snap_epoch': 76, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2024-04-24T18:08:29.994 DEBUG:teuthology.orchestra.run.smithi078:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool rbd 2024-04-24T18:08:29.995 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:30.000+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:30.093 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:30.101+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:30.112 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #30, image image_client.0-clone30 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent31 2024-04-24T18:08:30.140 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:30.140 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:30.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120987 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138873+0000", 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0052488400000000003, 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052827200000000003, 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120987 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138873+0000", 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138873+0000", 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138873+0000", 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138880+0000", 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.452e-06 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138886+0000", 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3749999999999999e-06 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138893+0000", 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5729999999999996e-06 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121179 4.1 4:85e44762:::rbd_data.1174787cbd24.0000000000000004:21 [read 198656~2048] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139456+0000", 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.004665776, 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0047783909999999999, 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "reached pg", 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121179 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139456+0000", 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139456+0000", 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139456+0000", 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139458+0000", 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.153e-06 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139464+0000", 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5809999999999996e-06 2024-04-24T18:08:30.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139468+0000", 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9509999999999999e-06 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144189+0000", 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.004721701 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120988 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138898+0000", 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0052239749999999996, 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0054157320000000004, 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120988 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138898+0000", 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138898+0000", 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138898+0000", 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138900+0000", 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138907+0000", 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3910000000000003e-06 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138910+0000", 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.782e-06 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121180 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139474+0000", 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0046477369999999999, 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00490742, 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121180 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139474+0000", 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139474+0000", 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139474+0000", 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139476+0000", 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3429999999999998e-06 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139482+0000", 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6130000000000003e-06 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139486+0000", 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1629999999999998e-06 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.153 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121268 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139993+0000", 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0041286530000000004, 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0044597700000000001, 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121268 2024-04-24T18:08:30.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139993+0000", 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139993+0000", 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139993+0000", 2024-04-24T18:08:30.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139995+0000", 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0770000000000002e-06 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.140001+0000", 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7309999999999998e-06 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.140005+0000", 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1049999999999997e-06 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120989 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138915+0000", 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0052073889999999998, 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056081409999999996, 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120989 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138915+0000", 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138915+0000", 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138914+0000", 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138917+0000", 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2500000000000001e-06 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138922+0000", 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.981e-06 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138926+0000", 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6500000000000002e-06 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121181 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139490+0000", 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.004631769, 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050915919999999998, 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.160 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121181 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139490+0000", 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139490+0000", 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139490+0000", 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139492+0000", 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1110000000000002e-06 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139497+0000", 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3979999999999998e-06 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139501+0000", 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2349999999999999e-06 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121269 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144272+0000", 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9998498, 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00036542900000000001, 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.163 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121269 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144272+0000", 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144272+0000", 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144272+0000", 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144276+0000", 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4070000000000003e-06 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144285+0000", 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7479999999999999e-06 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144290+0000", 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3149999999999999e-06 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120990 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138931+0000", 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0051912809999999998, 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057718350000000003, 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120990 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138931+0000", 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138931+0000", 2024-04-24T18:08:30.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138930+0000", 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138933+0000", 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.26e-06 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138938+0000", 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4559999999999999e-06 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138942+0000", 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2390000000000004e-06 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121182 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139504+0000", 2024-04-24T18:08:30.169 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0046176070000000001, 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052759089999999996, 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121182 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139504+0000", 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139504+0000", 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139504+0000", 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139506+0000", 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139512+0000", 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7960000000000001e-06 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139516+0000", 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6660000000000001e-06 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121270 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144296+0000", 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9998264, 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000550177, 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121270 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144296+0000", 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144296+0000", 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144295+0000", 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144298+0000", 2024-04-24T18:08:30.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8059999999999999e-06 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144304+0000", 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2179999999999996e-06 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144308+0000", 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.206e-06 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120991 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138946+0000", 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.005175654, 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0059649560000000004, 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120991 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138946+0000", 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138946+0000", 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138946+0000", 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138948+0000", 2024-04-24T18:08:30.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3199999999999998e-06 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138954+0000", 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5310000000000001e-06 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138961+0000", 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7619999999999998e-06 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121183 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139520+0000", 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0046020979999999998, 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.005456163, 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.179 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121183 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139520+0000", 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139520+0000", 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139520+0000", 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139526+0000", 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8850000000000001e-06 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139532+0000", 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4639999999999997e-06 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139536+0000", 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.279e-06 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121271 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144313+0000", 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9998088, 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00072924000000000003, 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.182 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121271 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144313+0000", 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144313+0000", 2024-04-24T18:08:30.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144313+0000", 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144315+0000", 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4449999999999999e-06 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144321+0000", 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:08:30.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144326+0000", 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3810000000000001e-06 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120992 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138965+0000", 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0051571209999999998, 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0061413259999999999, 2024-04-24T18:08:30.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120992 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138965+0000", 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138965+0000", 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138965+0000", 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138967+0000", 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.441e-06 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138973+0000", 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6269999999999997e-06 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138976+0000", 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5090000000000002e-06 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121184 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139540+0000", 2024-04-24T18:08:30.188 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0045816609999999999, 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056298540000000001, 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121184 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139540+0000", 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139540+0000", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139540+0000", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139542+0000", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4370000000000001e-06 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139548+0000", 2024-04-24T18:08:30.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4909999999999996e-06 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139552+0000", 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.703e-06 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121272 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144330+0000", 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9997921, 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00090667999999999999, 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.191 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121272 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144330+0000", 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144330+0000", 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144330+0000", 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144332+0000", 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3530000000000002e-06 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144339+0000", 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.72e-06 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144343+0000", 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.98e-06 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120993 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138980+0000", 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0051420490000000001, 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0063277810000000002, 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.194 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120993 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138980+0000", 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138980+0000", 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138980+0000", 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138982+0000", 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.035e-06 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138987+0000", 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5809999999999996e-06 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138993+0000", 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6239999999999999e-06 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121185 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139556+0000", 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0045663479999999996, 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0058195729999999998, 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121185 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139556+0000", 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139556+0000", 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139555+0000", 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139558+0000", 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3420000000000002e-06 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139563+0000", 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.468e-06 2024-04-24T18:08:30.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139567+0000", 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6210000000000001e-06 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121273 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144347+0000", 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9997749, 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011012070000000001, 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.200 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121273 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144347+0000", 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144347+0000", 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144347+0000", 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144349+0000", 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3520000000000001e-06 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144353+0000", 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4889999999999997e-06 2024-04-24T18:08:30.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144356+0000", 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2280000000000001e-06 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120994 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.138997+0000", 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.005124689, 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0065178359999999999, 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.203 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120994 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138997+0000", 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138997+0000", 2024-04-24T18:08:30.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138997+0000", 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.138999+0000", 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139005+0000", 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7699999999999998e-06 2024-04-24T18:08:30.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139008+0000", 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3050000000000001e-06 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121186 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139571+0000", 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0045511780000000003, 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0060131259999999997, 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121186 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139571+0000", 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139571+0000", 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139571+0000", 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139573+0000", 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1909999999999999e-06 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139578+0000", 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4330000000000003e-06 2024-04-24T18:08:30.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139582+0000", 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.529e-06 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121274 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144358+0000", 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.999764, 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001309431, 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.209 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121274 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144358+0000", 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144358+0000", 2024-04-24T18:08:30.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144358+0000", 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144359+0000", 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4959999999999999e-06 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144362+0000", 2024-04-24T18:08:30.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.055e-06 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144364+0000", 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.995e-06 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120995 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139013+0000", 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0051093340000000001, 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0067239700000000001, 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120995 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139013+0000", 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139013+0000", 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139012+0000", 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139015+0000", 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1440000000000001e-06 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139020+0000", 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8549999999999998e-06 2024-04-24T18:08:30.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139024+0000", 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4989999999999999e-06 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121188 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139586+0000", 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0045361519999999999, 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0062171919999999999, 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.215 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121188 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139586+0000", 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139586+0000", 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139586+0000", 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139588+0000", 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139593+0000", 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3680000000000001e-06 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139597+0000", 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6239999999999999e-06 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121275 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144367+0000", 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9997549, 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001501046, 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.218 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121275 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144367+0000", 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144367+0000", 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144367+0000", 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144368+0000", 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4470000000000001e-06 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144371+0000", 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0580000000000002e-06 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144374+0000", 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0499999999999999e-06 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120996 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139029+0000", 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0050928220000000003, 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0069106669999999997, 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.221 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120996 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139029+0000", 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139029+0000", 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139029+0000", 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139031+0000", 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1739999999999999e-06 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139037+0000", 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5890000000000002e-06 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139040+0000", 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.569e-06 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121190 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139608+0000", 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.004514461, 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0064013539999999997, 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121190 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139608+0000", 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139608+0000", 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139607+0000", 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139610+0000", 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.497e-06 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139617+0000", 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2450000000000001e-06 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139621+0000", 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9779999999999999e-06 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121276 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144375+0000", 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9997463, 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017074729999999999, 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.227 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121276 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144375+0000", 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144375+0000", 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144375+0000", 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144387+0000", 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1595e-05 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144390+0000", 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5810000000000001e-06 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144393+0000", 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6800000000000002e-06 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120997 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139060+0000", 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0050619360000000004, 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0070888790000000002, 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.230 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120997 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139060+0000", 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139060+0000", 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139060+0000", 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139062+0000", 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6450000000000001e-06 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139067+0000", 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.305e-06 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139071+0000", 2024-04-24T18:08:30.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9299999999999996e-06 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121192 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139625+0000", 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0044967510000000002, 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0065920099999999997, 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.233 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121192 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139625+0000", 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139625+0000", 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139625+0000", 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139627+0000", 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139633+0000", 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5890000000000002e-06 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139636+0000", 2024-04-24T18:08:30.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2799999999999999e-06 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121277 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144396+0000", 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9997263, 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0018869589999999999, 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121277 2024-04-24T18:08:30.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144396+0000", 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144396+0000", 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144395+0000", 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144397+0000", 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4309999999999999e-06 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144400+0000", 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9510000000000002e-06 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144402+0000", 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9439999999999999e-06 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.238 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120998 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139077+0000", 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0050452609999999997, 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0072782089999999999, 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120998 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139077+0000", 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139077+0000", 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139076+0000", 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139079+0000", 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.79e-06 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139085+0000", 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7989999999999999e-06 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139088+0000", 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.551e-06 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121194 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139640+0000", 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0044819509999999996, 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0067860519999999999, 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121194 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139640+0000", 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139640+0000", 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139640+0000", 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139642+0000", 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4080000000000001e-06 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139648+0000", 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5130000000000004e-06 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139651+0000", 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6169999999999998e-06 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.244 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121278 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144404+0000", 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9997182, 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020961209999999998, 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121278 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144404+0000", 2024-04-24T18:08:30.247 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:30.254+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144404+0000", 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144404+0000", 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144405+0000", 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.598e-06 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144408+0000", 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.993e-06 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144410+0000", 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.922e-06 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.249 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:120999 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139092+0000", 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0050296020000000002, 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074869109999999997, 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 120999 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.250 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139092+0000", 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139092+0000", 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139092+0000", 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139103+0000", 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.079e-05 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139109+0000", 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7030000000000003e-06 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139113+0000", 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0600000000000001e-06 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121196 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139657+0000", 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0044653610000000002, 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0069810779999999999, 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121196 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139657+0000", 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139657+0000", 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139656+0000", 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139659+0000", 2024-04-24T18:08:30.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139664+0000", 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5230000000000003e-06 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139668+0000", 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.997e-06 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121279 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.255 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144412+0000", 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9997096, 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0022961919999999999, 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121279 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144412+0000", 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144412+0000", 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144412+0000", 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144413+0000", 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3519999999999999e-06 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144417+0000", 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0649999999999999e-06 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144419+0000", 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1409999999999999e-06 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121000 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.258 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139136+0000", 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0049857870000000002, 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0076379610000000004, 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121000 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139136+0000", 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139136+0000", 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139136+0000", 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139139+0000", 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.858e-06 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139145+0000", 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3130000000000004e-06 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139150+0000", 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6489999999999999e-06 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121198 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139673+0000", 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0044485890000000002, 2024-04-24T18:08:30.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0071676140000000001, 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121198 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139673+0000", 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139673+0000", 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139673+0000", 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139675+0000", 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1560000000000002e-06 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139681+0000", 2024-04-24T18:08:30.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6119999999999998e-06 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139685+0000", 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7009999999999998e-06 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121280 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144421+0000", 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.999701, 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002489762, 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.264 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121280 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144421+0000", 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144421+0000", 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144421+0000", 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144422+0000", 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2139999999999999e-06 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144425+0000", 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1549999999999999e-06 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144427+0000", 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.962e-06 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121001 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139155+0000", 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0049669939999999997, 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007823666, 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121001 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139155+0000", 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139155+0000", 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139155+0000", 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139157+0000", 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2400000000000002e-06 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139163+0000", 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9529999999999996e-06 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139166+0000", 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.72e-06 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121200 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139689+0000", 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0044334910000000003, 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073530929999999998, 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.270 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121200 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139689+0000", 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139689+0000", 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139688+0000", 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139690+0000", 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1789999999999998e-06 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139696+0000", 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4380000000000003e-06 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139699+0000", 2024-04-24T18:08:30.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.653e-06 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121281 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144429+0000", 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9996929, 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00268315, 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121281 2024-04-24T18:08:30.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144429+0000", 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144429+0000", 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144429+0000", 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144430+0000", 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.235e-06 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144433+0000", 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8660000000000002e-06 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144435+0000", 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0140000000000001e-06 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121002 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139170+0000", 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0049515640000000003, 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080056659999999998, 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121002 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139170+0000", 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139170+0000", 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139170+0000", 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139172+0000", 2024-04-24T18:08:30.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.306e-06 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139178+0000", 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5790000000000003e-06 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139182+0000", 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.155e-06 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.278 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121202 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139703+0000", 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00441874, 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0075399400000000002, 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121202 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139703+0000", 2024-04-24T18:08:30.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139703+0000", 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139703+0000", 2024-04-24T18:08:30.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139705+0000", 2024-04-24T18:08:30.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2730000000000001e-06 2024-04-24T18:08:30.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139714+0000", 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139718+0000", 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3589999999999999e-06 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121282 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144437+0000", 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9996853, 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028732079999999999, 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121282 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144437+0000", 2024-04-24T18:08:30.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144437+0000", 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144437+0000", 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144438+0000", 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2309999999999999e-06 2024-04-24T18:08:30.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144441+0000", 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9579999999999999e-06 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144443+0000", 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.339e-06 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121004 4.13 4.3c01dbb3 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.286 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139187+0000", 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0049349019999999997, 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.008189956, 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121004 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139187+0000", 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139187+0000", 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139187+0000", 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139189+0000", 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2189999999999998e-06 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139194+0000", 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3319999999999999e-06 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139199+0000", 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4259999999999996e-06 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121204 4.1 4.46e227a1 (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.139722+0000", 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0043997810000000002, 2024-04-24T18:08:30.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0077230090000000003, 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121204 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139722+0000", 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139722+0000", 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139721+0000", 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139724+0000", 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9689999999999999e-06 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139730+0000", 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7529999999999998e-06 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.139734+0000", 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7890000000000001e-06 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.292 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121283 4.1b 4.596d1cbb (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.144446+0000", 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9996762, 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030614320000000002, 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121283 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144446+0000", 2024-04-24T18:08:30.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144446+0000", 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144445+0000", 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144447+0000", 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.367e-06 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144450+0000", 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0850000000000001e-06 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.144452+0000", 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.888e-06 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121412 4.e 4:73f1e9b9:::rbd_data.1174787cbd24.000000000000000f:21 [read 51200~4096] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.145534+0000", 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9985881, 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020170470000000001, 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "reached pg", 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121412 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145534+0000", 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145534+0000", 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145534+0000", 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145536+0000", 2024-04-24T18:08:30.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.486e-06 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145542+0000", 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.767e-06 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145545+0000", 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.399e-06 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.147330+0000", 2024-04-24T18:08:30.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017845719999999999 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121414 4.e 4.9d978fce (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.145565+0000", 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9985566, 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020679470000000001, 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121414 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145565+0000", 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145565+0000", 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145565+0000", 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145567+0000", 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3439999999999999e-06 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145573+0000", 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5319999999999997e-06 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145577+0000", 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.597e-06 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121416 4.e 4.9d978fce (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.145596+0000", 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9985256, 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021110669999999999, 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.302 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121416 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145596+0000", 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145596+0000", 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145596+0000", 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145598+0000", 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1689999999999999e-06 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145608+0000", 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9469999999999998e-06 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145613+0000", 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6890000000000003e-06 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:121418 4.e 4.9d978fce (undecoded) ondisk+retry+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.145633+0000", 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4294967295.9984894, 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.002142499, 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.305 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 121418 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145633+0000", 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145633+0000", 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145632+0000", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145634+0000", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.215e-06 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145640+0000", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3059999999999997e-06 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.145643+0000", 2024-04-24T18:08:30.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.507e-06 2024-04-24T18:08:30.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.308 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.308 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-24T18:08:30.308 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 54 2024-04-24T18:08:30.308 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:30.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:30.351 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:30.360+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:30.430 INFO:teuthology.orchestra.run.smithi049.stdout:3469 trunc from 0xc2dad70 to 0x5e20da3 2024-04-24T18:08:30.448 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:30.457+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:30.449 INFO:teuthology.orchestra.run.smithi049.stdout:3470 read 0x52182af thru 0x521d4fd (0x524f bytes) 2024-04-24T18:08:30.449 INFO:teuthology.orchestra.run.smithi049.stdout:3471 read 0x3f62a64 thru 0x3f63a88 (0x1025 bytes) 2024-04-24T18:08:30.449 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:30.463 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:30.463 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:30.463 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:30.463 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:30.463 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:30.464 INFO:teuthology.orchestra.run.smithi049.stdout:3473 write 0xdda70f8 thru 0xddb155d (0xa466 bytes) 2024-04-24T18:08:30.467 INFO:teuthology.orchestra.run.smithi049.stdout:3474 read 0xa1d4c77 thru 0xa1d6d51 (0x20db bytes) 2024-04-24T18:08:30.467 INFO:teuthology.orchestra.run.smithi049.stdout:3475 punch from 0x4291ea3 to 0x429b088, (0x91e5 bytes) 2024-04-24T18:08:30.468 INFO:teuthology.orchestra.run.smithi049.stdout:3476 write 0x173df44 thru 0x174cbdc (0xec99 bytes) 2024-04-24T18:08:30.474 INFO:teuthology.orchestra.run.smithi049.stdout:3477 punch from 0x811868 to 0x81d6be, (0xbe56 bytes) 2024-04-24T18:08:30.474 INFO:teuthology.orchestra.run.smithi049.stdout:3478 read 0x61ee10a thru 0x61fa6a7 (0xc59e bytes) 2024-04-24T18:08:30.474 INFO:teuthology.orchestra.run.smithi049.stdout:3479 clone 33 order 22 su 32768 sc 15 2024-04-24T18:08:30.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:30.549 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:30.557+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:30.642 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:30.642 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:30.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:30.642 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:30.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.642 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11780 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2666496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.642 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.313189432999998, 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042348956, 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11780 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335768+0000", 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1910000000000003e-06 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335774+0000", 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9089999999999997e-06 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335778+0000", 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9339999999999999e-06 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378028+0000", 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042250303000000003 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378072+0000", 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3909000000000003e-05 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378110+0000", 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8050000000000003e-05 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.646 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11792 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.311176637000003, 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042313245999999999, 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11792 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.648 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.649 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.649 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:30.649 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.649 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.649 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337773+0000", 2024-04-24T18:08:30.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337781+0000", 2024-04-24T18:08:30.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3089999999999998e-06 2024-04-24T18:08:30.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337787+0000", 2024-04-24T18:08:30.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.285e-06 2024-04-24T18:08:30.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337791+0000", 2024-04-24T18:08:30.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.065e-06 2024-04-24T18:08:30.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.379933+0000", 2024-04-24T18:08:30.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042141913000000003 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380016+0000", 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2959000000000003e-05 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380087+0000", 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1038000000000007e-05 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.654 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11799 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2856960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.655 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:30.655 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.309939288999999, 2024-04-24T18:08:30.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042339663999999999, 2024-04-24T18:08:30.655 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.655 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.655 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11799 2024-04-24T18:08:30.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.656 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:30.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:30.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339013+0000", 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339019+0000", 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339023+0000", 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7689999999999998e-06 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381202+0000", 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042179054000000001 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381285+0000", 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3299000000000003e-05 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381351+0000", 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5795000000000003e-05 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.309923509000001, 2024-04-24T18:08:30.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:30.664 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.309907602999999, 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:30.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:08:30.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.668 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.309855636000002, 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:08:30.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:08:30.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11805 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2920448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.308948936, 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042290415999999997, 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.673 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11805 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340003+0000", 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.514e-06 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340010+0000", 2024-04-24T18:08:30.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340014+0000", 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.805e-06 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382196+0000", 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042182175000000002 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382236+0000", 2024-04-24T18:08:30.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0265999999999997e-05 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382292+0000", 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5791000000000002e-05 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.308932161999998, 2024-04-24T18:08:30.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.30891553, 2024-04-24T18:08:30.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:08:30.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.685 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11809 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2965504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.308232077, 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042364339000000001, 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11809 2024-04-24T18:08:30.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:30.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340721+0000", 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1370000000000002e-06 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340727+0000", 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.533e-06 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340732+0000", 2024-04-24T18:08:30.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5329999999999996e-06 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382924+0000", 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042192039000000001 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383012+0000", 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7669000000000004e-05 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383083+0000", 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0988999999999994e-05 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.308213926999997, 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.692 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.308197425000003, 2024-04-24T18:08:30.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:30.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:08:30.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:08:30.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11814 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3012608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.307302843000002, 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042354992000000001, 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11814 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341648+0000", 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.372e-06 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341652+0000", 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0690000000000002e-06 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341654+0000", 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0559999999999999e-06 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383852+0000", 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042198125000000003 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383934+0000", 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2021999999999995e-05 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384002+0000", 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8669000000000003e-05 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.307294558999999, 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.307286533999999, 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.712 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11819 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3065856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.306254825000003, 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042208806000000001, 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11819 2024-04-24T18:08:30.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:30.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342698+0000", 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6879999999999999e-06 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342704+0000", 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0090000000000004e-06 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342708+0000", 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9029999999999997e-06 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384745+0000", 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042037243000000002 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384833+0000", 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7769999999999995e-05 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384904+0000", 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1632999999999997e-05 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.717 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11820 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3076096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.306238161000003, 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042328461999999997, 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11820 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342714+0000", 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342720+0000", 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8479999999999997e-06 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342724+0000", 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6430000000000001e-06 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384944+0000", 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042220698000000001 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384992+0000", 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7441999999999997e-05 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385041+0000", 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8715999999999999e-05 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.721 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.306222392000002, 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.722 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:30.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:08:30.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:30.726 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.306206422999999, 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:08:30.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:08:30.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:08:30.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.2460079840000002, 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:30.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:30.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:08:30.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:08:30.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:30.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:30.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:30.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:30.738 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:30.740 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:30.740 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:30.747+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:30.790 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:08:30.844 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:30.852+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:30.872 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:30.873 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:30.873 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:30.873 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:30.883 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:30.908 INFO:teuthology.orchestra.run.smithi049.stdout:leaving image image_client.0-clone32 intact 2024-04-24T18:08:30.941 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:30.949+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:31.006 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:31.006 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:31.006 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:31.006 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:31.006 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:31.016 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:31.042 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:31.050+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:31.085 INFO:teuthology.orchestra.run.smithi078.stderr:Nothing to do for non-filestore backend 2024-04-24T18:08:31.143 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:31.151+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:31.188 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:31.188 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:31.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:31.188 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:31.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.188 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.188 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83164813500000001, 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:31.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.192 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83164000599999999, 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:08:31.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.197 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83163198299999996, 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:31.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:08:31.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:08:31.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83162307000000002, 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:31.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.206 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83161425099999997, 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.207 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.831605119, 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:08:31.211 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:08:31.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83078485999999996, 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:08:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:08:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.218 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:08:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:08:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:08:31.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83075766100000004, 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:08:31.221 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:08:31.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.225 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83074019099999996, 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83072485100000004, 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.230 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:31.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:08:31.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.234 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.830700773, 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:31.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:08:31.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:08:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:08:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:08:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:08:31.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83068520800000001, 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:08:31.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:08:31.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83067024700000003, 2024-04-24T18:08:31.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:08:31.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:08:31.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83065330800000003, 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:08:31.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83063690300000004, 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.830620464, 2024-04-24T18:08:31.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:31.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:08:31.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:08:31.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83060562400000004, 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:08:31.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83058920000000003, 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:08:31.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:08:31.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83057198499999996, 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.272 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:08:31.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:08:31.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.276 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.83055451999999996, 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:31.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:08:31.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:08:31.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.281 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:31.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:31.284 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:31.291+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #31, image image_client.0-clone31 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent32 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:3481 write 0x2ad21c2 thru 0x2ae02a8 (0xe0e7 bytes) 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:3482 write 0xc5d52f3 thru 0xc5daa1a (0x5728 bytes) 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:3483 write 0x4217a46 thru 0x421b3ec (0x39a7 bytes) 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:3486 writesame 0x2daebc2 thru 0x2dbd952 data_size 0x1(0xed91 bytes) 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:3487 read 0x413cca8 thru 0x4146678 (0x99d1 bytes) 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:3489 read 0x6b823ff thru 0x6b855ad (0x31af bytes) 2024-04-24T18:08:31.285 INFO:teuthology.orchestra.run.smithi049.stdout:3490 write 0xdb7e992 thru 0xdb80c9d (0x230c bytes) 2024-04-24T18:08:31.289 INFO:teuthology.orchestra.run.smithi049.stdout:3491 trunc from 0xddb155e to 0x4b7ace6 2024-04-24T18:08:31.346 INFO:teuthology.orchestra.run.smithi049.stdout:3492 trunc from 0x4b7ace6 to 0x5e5ad7d 2024-04-24T18:08:31.351 INFO:teuthology.orchestra.run.smithi049.stdout:3494 punch from 0x3a4d946 to 0x3a5b51b, (0xdbd5 bytes) 2024-04-24T18:08:31.351 INFO:teuthology.orchestra.run.smithi049.stdout:3495 read 0x43b0f4a thru 0x43b21b3 (0x126a bytes) 2024-04-24T18:08:31.366 INFO:teuthology.orchestra.run.smithi049.stdout:3496 write 0xeb66e56 thru 0xeb726f3 (0xb89e bytes) 2024-04-24T18:08:31.371 INFO:teuthology.orchestra.run.smithi049.stdout:3498 writesame 0x88ad238 thru 0x88b09a2 data_size 0x1(0x376b bytes) 2024-04-24T18:08:31.378 INFO:teuthology.orchestra.run.smithi049.stdout:3500 punch from 0xd834154 to 0xd843e97, (0xfd43 bytes) 2024-04-24T18:08:31.381 INFO:teuthology.orchestra.run.smithi049.stdout:3501 trunc from 0xeb726f4 to 0xc2d1d89 2024-04-24T18:08:31.383 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:31.392+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:31.388 INFO:teuthology.orchestra.run.smithi049.stdout:3503 read 0xb1eaa66 thru 0xb1ee266 (0x3801 bytes) 2024-04-24T18:08:31.388 INFO:teuthology.orchestra.run.smithi049.stdout:3504 read 0x57c32b0 thru 0x57c3bf7 (0x948 bytes) 2024-04-24T18:08:31.388 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:31.389 INFO:teuthology.orchestra.run.smithi049.stdout:3506 trunc from 0xc2d1d89 to 0xd26d46e 2024-04-24T18:08:31.394 INFO:teuthology.orchestra.run.smithi049.stdout:3508 write 0xbbedcd thru 0xbcdcff (0xef33 bytes) 2024-04-24T18:08:31.419 INFO:teuthology.orchestra.run.smithi049.stdout:3509 write 0x654cbc5 thru 0x6557a15 (0xae51 bytes) 2024-04-24T18:08:31.421 INFO:teuthology.orchestra.run.smithi049.stdout:3510 write 0x405c649 thru 0x406c08d (0xfa45 bytes) 2024-04-24T18:08:31.424 INFO:teuthology.orchestra.run.smithi049.stdout:3513 trunc from 0xd26d46e to 0xe4694e2 2024-04-24T18:08:31.433 INFO:teuthology.orchestra.run.smithi049.stdout:3514 write 0xe466705 thru 0xe46e086 (0x7982 bytes) 2024-04-24T18:08:31.434 INFO:teuthology.orchestra.run.smithi049.stdout:3515 clone 34 order 23 su 4194304 sc 15 2024-04-24T18:08:31.454 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:31.454 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:31.455 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:31.455 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:31.465 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:31.484 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:31.493+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:31.586 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:31.593+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:31.615 DEBUG:teuthology.orchestra.run.smithi078:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool .mgr 2024-04-24T18:08:31.631 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:31.631 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:31.631 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:31.631 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:31.631 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:31.642 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:31.686 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:31.694+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:31.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:31.787 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:31.795+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:31.813 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:31.813 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:31.813 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:31.813 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202549505999997, 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:08:31.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202480453999996, 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202456038999998, 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:08:31.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:08:31.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:08:31.830 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202432889999997, 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:08:31.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202352321999996, 2024-04-24T18:08:31.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.838 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.841 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202327511999997, 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.842 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:08:31.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:08:31.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:08:31.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202307399999995, 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:08:31.847 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202236737000007, 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 71.202224963999996, 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.858 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:31.863 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.781903134999993, 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:08:31.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:08:31.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.868 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.781855953000004, 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.869 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:08:31.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:08:31.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:08:31.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.781830831999997, 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:08:31.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:31.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:08:31.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:08:31.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.879 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.781778697999997, 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.781715234000004, 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.885 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:31.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:08:31.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:08:31.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:08:31.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.890 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:156 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.270544935000004, 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0073643390000000001, 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 156 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:31.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548230+0000", 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548239+0000", 2024-04-24T18:08:31.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8899999999999996e-06 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548246+0000", 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9370000000000004e-06 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548259+0000", 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549586+0000", 2024-04-24T18:08:31.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001326743 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549641+0000", 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5856999999999997e-05 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.549713+0000", 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1970000000000004e-05 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550899+0000", 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001185567 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555532+0000", 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0046325760000000002 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555573+0000", 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1730000000000002e-05 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555595+0000", 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1324000000000001e-05 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.270509562000001, 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:31.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.270484534999994, 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:31.907 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 70.270405298, 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.912 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 39.008216517000001, 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:31.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:08:31.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:08:31.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:08:31.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 39.007110447999999, 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:31.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:31.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:31.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:31.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:08:31.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:08:31.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.921 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:31.921 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:31.921 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:31.922 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone33 from 0xe46e087 (overlap 0x4b7ace6) to 0x0 2024-04-24T18:08:31.922 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:31.930+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:31.923 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:31.929 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:31.969 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:31.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:32.013 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:32.023 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:32.031+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:32.035 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:32.057 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:32.080 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:32.081 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:32.084 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:32.084 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:32.095 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:32.095 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:32.107 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:32.148 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:32.156+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:32.193 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #32, image image_client.0-clone32 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent33 2024-04-24T18:08:32.226 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:32.234+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:32.296 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:32.296 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:32.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:32.327 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:32.335+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:32.428 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:32.436+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:32.472 INFO:teuthology.orchestra.run.smithi049.stdout:3516 write 0x89d0273 thru 0x89d93b3 (0x9141 bytes) 2024-04-24T18:08:32.481 INFO:teuthology.orchestra.run.smithi049.stdout:3517 read 0x709b843 thru 0x70a5d71 (0xa52f bytes) 2024-04-24T18:08:32.481 INFO:teuthology.orchestra.run.smithi049.stdout:3519 writesame 0xe260d29 thru 0xe2697a3 data_size 0x27(0x8a7b bytes) 2024-04-24T18:08:32.484 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:32.484 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:32.484 INFO:teuthology.orchestra.run.smithi049.stdout:3522 read 0x7501336 thru 0x7508450 (0x711b bytes) 2024-04-24T18:08:32.485 INFO:teuthology.orchestra.run.smithi049.stdout:3523 trunc from 0xe46e087 to 0xea7f2a4 2024-04-24T18:08:32.490 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.491 INFO:teuthology.orchestra.run.smithi049.stdout:3526 write 0x12ad800 thru 0x12b6ca6 (0x94a7 bytes) 2024-04-24T18:08:32.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:32.498 INFO:teuthology.orchestra.run.smithi049.stdout:3527 writesame 0x7f54aca thru 0x7f5679c data_size 0x1(0x1cd3 bytes) 2024-04-24T18:08:32.504 INFO:teuthology.orchestra.run.smithi049.stdout:3529 write 0x66e568b thru 0x66f0a83 (0xb3f9 bytes) 2024-04-24T18:08:32.520 INFO:teuthology.orchestra.run.smithi049.stdout:3530 punch from 0x1f7f98b to 0x1f8b0a3, (0xb718 bytes) 2024-04-24T18:08:32.520 INFO:teuthology.orchestra.run.smithi049.stdout:3532 write 0x20b7d27 thru 0x20c06ec (0x89c6 bytes) 2024-04-24T18:08:32.526 INFO:teuthology.orchestra.run.smithi049.stdout:3536 read 0x6737f80 thru 0x673b0ce (0x314f bytes) 2024-04-24T18:08:32.529 INFO:teuthology.orchestra.run.smithi049.stdout:3537 write 0x7f16ba9 thru 0x7f225b2 (0xba0a bytes) 2024-04-24T18:08:32.530 INFO:teuthology.orchestra.run.smithi049.stdout:3538 punch from 0x1ddeb2b to 0x1de62bd, (0x7792 bytes) 2024-04-24T18:08:32.530 INFO:teuthology.orchestra.run.smithi049.stdout:3541 write 0x281b243 thru 0x281e0f4 (0x2eb2 bytes) 2024-04-24T18:08:32.530 INFO:teuthology.orchestra.run.smithi049.stdout:3542 read 0xd3770b7 thru 0xd37970c (0x2656 bytes) 2024-04-24T18:08:32.531 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:32.538+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:32.531 INFO:teuthology.orchestra.run.smithi049.stdout:3543 punch from 0xa3fdb79 to 0xa402ac3, (0x4f4a bytes) 2024-04-24T18:08:32.531 INFO:teuthology.orchestra.run.smithi049.stdout:3545 read 0x20f7615 thru 0x20fc22d (0x4c19 bytes) 2024-04-24T18:08:32.531 INFO:teuthology.orchestra.run.smithi049.stdout:3546 write 0xc844497 thru 0xc84f6a0 (0xb20a bytes) 2024-04-24T18:08:32.535 INFO:teuthology.orchestra.run.smithi049.stdout:3547 write 0xb7d4301 thru 0xb7d946d (0x516d bytes) 2024-04-24T18:08:32.538 INFO:teuthology.orchestra.run.smithi049.stdout:3549 write 0xcd4f425 thru 0xcd5a50a (0xb0e6 bytes) 2024-04-24T18:08:32.542 INFO:teuthology.orchestra.run.smithi049.stdout:3550 writesame 0x11ce1ae thru 0x11d86e4 data_size 0xb(0xa537 bytes) 2024-04-24T18:08:32.544 INFO:teuthology.orchestra.run.smithi049.stdout:3551 write 0x3928ccd thru 0x39360aa (0xd3de bytes) 2024-04-24T18:08:32.552 INFO:teuthology.orchestra.run.smithi049.stdout:3554 write 0x5ab2d7a thru 0x5abda2c (0xacb3 bytes) 2024-04-24T18:08:32.559 INFO:teuthology.orchestra.run.smithi049.stdout:3555 read 0xb16efd1 thru 0xb1738d6 (0x4906 bytes) 2024-04-24T18:08:32.559 INFO:teuthology.orchestra.run.smithi049.stdout:3556 punch from 0x6965da0 to 0x6968acc, (0x2d2c bytes) 2024-04-24T18:08:32.559 INFO:teuthology.orchestra.run.smithi049.stdout:3557 punch from 0x258f9c2 to 0x25961c6, (0x6804 bytes) 2024-04-24T18:08:32.559 INFO:teuthology.orchestra.run.smithi049.stdout:3558 write 0x6f30811 thru 0x6f3f894 (0xf084 bytes) 2024-04-24T18:08:32.565 INFO:teuthology.orchestra.run.smithi049.stdout:3561 write 0x3b06f45 thru 0x3b10ccf (0x9d8b bytes) 2024-04-24T18:08:32.578 INFO:teuthology.orchestra.run.smithi049.stdout:3562 trunc from 0xea7f2a4 to 0x454470a 2024-04-24T18:08:32.626 INFO:teuthology.orchestra.run.smithi049.stdout:3563 punch from 0x296decb to 0x29727fd, (0x4932 bytes) 2024-04-24T18:08:32.627 INFO:teuthology.orchestra.run.smithi049.stdout:3564 trunc from 0x454470a to 0x478e020 2024-04-24T18:08:32.631 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:32.639+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:32.633 INFO:teuthology.orchestra.run.smithi049.stdout:3565 write 0x115b10e thru 0x116a677 (0xf56a bytes) 2024-04-24T18:08:32.649 INFO:teuthology.orchestra.run.smithi049.stdout:3566 write 0xd1107af thru 0xd113635 (0x2e87 bytes) 2024-04-24T18:08:32.652 INFO:teuthology.orchestra.run.smithi049.stdout:3567 write 0x7a119f1 thru 0x7a1e6c7 (0xccd7 bytes) 2024-04-24T18:08:32.655 INFO:teuthology.orchestra.run.smithi049.stdout:3571 write 0x55c094f thru 0x55c6404 (0x5ab6 bytes) 2024-04-24T18:08:32.656 INFO:teuthology.orchestra.run.smithi049.stdout:3572 read 0x2e7d454 thru 0x2e8399b (0x6548 bytes) 2024-04-24T18:08:32.657 INFO:teuthology.orchestra.run.smithi049.stdout:3575 read 0x1f689ee thru 0x1f6d965 (0x4f78 bytes) 2024-04-24T18:08:32.657 INFO:teuthology.orchestra.run.smithi049.stdout:3576 write 0x444371a thru 0x4447228 (0x3b0f bytes) 2024-04-24T18:08:32.663 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:32.664 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:32.667 INFO:teuthology.orchestra.run.smithi049.stdout:3577 trunc from 0xd113636 to 0x5ba11e7 2024-04-24T18:08:32.678 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:32.679 INFO:teuthology.orchestra.run.smithi049.stdout:3578 punch from 0x121935b to 0x121a094, (0xd39 bytes) 2024-04-24T18:08:32.679 INFO:teuthology.orchestra.run.smithi049.stdout:3580 write 0x3507be2 thru 0x35127ff (0xac1e bytes) 2024-04-24T18:08:32.680 INFO:teuthology.orchestra.run.smithi049.stdout:3581 punch from 0x34b52e0 to 0x34c0efb, (0xbc1b bytes) 2024-04-24T18:08:32.680 INFO:teuthology.orchestra.run.smithi049.stdout:3582 punch from 0x1336c1 to 0x13aa0c, (0x734b bytes) 2024-04-24T18:08:32.687 INFO:teuthology.orchestra.run.smithi049.stdout:3584 write 0x78ce566 thru 0x78daa5f (0xc4fa bytes) 2024-04-24T18:08:32.690 INFO:teuthology.orchestra.run.smithi049.stdout:3585 writesame 0x3413bc0 thru 0x3418f08 data_size 0x9(0x5349 bytes) 2024-04-24T18:08:32.692 INFO:teuthology.orchestra.run.smithi049.stdout:3586 read 0x172af7 thru 0x178f3c (0x6446 bytes) 2024-04-24T18:08:32.694 INFO:teuthology.orchestra.run.smithi078.stderr:Nothing to do for non-filestore backend 2024-04-24T18:08:32.706 INFO:teuthology.orchestra.run.smithi049.stdout:3587 write 0xe599b2a thru 0xe5a4904 (0xaddb bytes) 2024-04-24T18:08:32.708 INFO:teuthology.orchestra.run.smithi049.stdout:3588 punch from 0xd4e1d02 to 0xd4e33de, (0x16dc bytes) 2024-04-24T18:08:32.708 INFO:teuthology.orchestra.run.smithi049.stdout:3589 write 0x1122f81 thru 0x112e5aa (0xb62a bytes) 2024-04-24T18:08:32.710 INFO:teuthology.orchestra.run.smithi049.stdout:3591 writesame 0x81d56af thru 0x81e12d0 data_size 0x2e(0xbc22 bytes) 2024-04-24T18:08:32.713 INFO:teuthology.orchestra.run.smithi049.stdout:3592 read 0x1b415f4 thru 0x1b4e193 (0xcba0 bytes) 2024-04-24T18:08:32.713 INFO:teuthology.orchestra.run.smithi049.stdout:3593 write 0xb8c27b5 thru 0xb8c7362 (0x4bae bytes) 2024-04-24T18:08:32.717 INFO:teuthology.orchestra.run.smithi049.stdout:3595 punch from 0x683fe59 to 0x68436fe, (0x38a5 bytes) 2024-04-24T18:08:32.717 INFO:teuthology.orchestra.run.smithi049.stdout:3596 read 0x9c0c62 thru 0x9c7d59 (0x70f8 bytes) 2024-04-24T18:08:32.717 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.729 INFO:teuthology.orchestra.run.smithi049.stdout:3599 trunc from 0xe5a4905 to 0x40ae846 2024-04-24T18:08:32.732 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:32.740+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:32.760 INFO:teuthology.orchestra.run.smithi049.stdout:3600 trunc from 0x40ae846 to 0x24deaa 2024-04-24T18:08:32.803 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:08:32.812 INFO:teuthology.orchestra.run.smithi049.stdout:3601 write 0x7e57381 thru 0x7e5cb4a (0x57ca bytes) 2024-04-24T18:08:32.820 INFO:teuthology.orchestra.run.smithi049.stdout:3602 trunc from 0x7e5cb4b to 0x38fcb5b 2024-04-24T18:08:32.833 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:32.841+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:32.841 INFO:teuthology.orchestra.run.smithi049.stdout:3604 writesame 0x61724b1 thru 0x617ede1 data_size 0x1(0xc931 bytes) 2024-04-24T18:08:32.843 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:32.843 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:32.856 INFO:teuthology.orchestra.run.smithi049.stdout:3605 write 0xdcbac8a thru 0xdcbf729 (0x4aa0 bytes) 2024-04-24T18:08:32.859 INFO:teuthology.orchestra.run.smithi049.stdout:3606 read 0x6d13450 thru 0x6d1af00 (0x7ab1 bytes) 2024-04-24T18:08:32.859 INFO:teuthology.orchestra.run.smithi049.stdout:3607 write 0xd572eeb thru 0xd57e4fe (0xb614 bytes) 2024-04-24T18:08:32.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:32.863 INFO:teuthology.orchestra.run.smithi049.stdout:3608 read 0x98e03c0 thru 0x98ec23c (0xbe7d bytes) 2024-04-24T18:08:32.863 INFO:teuthology.orchestra.run.smithi049.stdout:3610 punch from 0x4c0cd96 to 0x4c18906, (0xbb70 bytes) 2024-04-24T18:08:32.863 INFO:teuthology.orchestra.run.smithi049.stdout:3611 read 0x3ff8877 thru 0x3ffc2a8 (0x3a32 bytes) 2024-04-24T18:08:32.863 INFO:teuthology.orchestra.run.smithi049.stdout:3613 write 0x146c73 thru 0x1491c0 (0x254e bytes) 2024-04-24T18:08:32.865 INFO:teuthology.orchestra.run.smithi049.stdout:3614 writesame 0xbd4409c thru 0xbd4e7dc data_size 0x1(0xa741 bytes) 2024-04-24T18:08:32.881 INFO:teuthology.orchestra.run.smithi049.stdout:3615 punch from 0x8adbf04 to 0x8ae75a9, (0xb6a5 bytes) 2024-04-24T18:08:32.881 INFO:teuthology.orchestra.run.smithi049.stdout:3616 write 0xa97100e thru 0xa971b1f (0xb12 bytes) 2024-04-24T18:08:32.884 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.884 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.884 INFO:teuthology.orchestra.run.smithi049.stdout:3620 read 0x9151527 thru 0x9154cfa (0x37d4 bytes) 2024-04-24T18:08:32.884 INFO:teuthology.orchestra.run.smithi049.stdout:3621 read 0xb009670 thru 0xb015b50 (0xc4e1 bytes) 2024-04-24T18:08:32.885 INFO:teuthology.orchestra.run.smithi049.stdout:3622 read 0xcf9be8 thru 0xcfd937 (0x3d50 bytes) 2024-04-24T18:08:32.885 INFO:teuthology.orchestra.run.smithi049.stdout:3623 write 0x210a7cf thru 0x211712e (0xc960 bytes) 2024-04-24T18:08:32.887 INFO:teuthology.orchestra.run.smithi049.stdout:3624 writesame 0x37c62a8 thru 0x37c7e63 data_size 0xa(0x1bbc bytes) 2024-04-24T18:08:32.891 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.891 INFO:teuthology.orchestra.run.smithi049.stdout:3627 read 0x6ccb384 thru 0x6cce5b9 (0x3236 bytes) 2024-04-24T18:08:32.891 INFO:teuthology.orchestra.run.smithi049.stdout:3628 read 0x3326974 thru 0x332ecd4 (0x8361 bytes) 2024-04-24T18:08:32.891 INFO:teuthology.orchestra.run.smithi049.stdout:3630 read 0xadc424f thru 0xadccc90 (0x8a42 bytes) 2024-04-24T18:08:32.891 INFO:teuthology.orchestra.run.smithi049.stdout:3631 punch from 0x7c419cd to 0x7c45e03, (0x4436 bytes) 2024-04-24T18:08:32.892 INFO:teuthology.orchestra.run.smithi049.stdout:3632 writesame 0x320cd26 thru 0x320fc94 data_size 0x1(0x2f6f bytes) 2024-04-24T18:08:32.898 INFO:teuthology.orchestra.run.smithi049.stdout:3633 writesame 0xdd8f39 thru 0xddfc65 data_size 0x1(0x6d2d bytes) 2024-04-24T18:08:32.913 INFO:teuthology.orchestra.run.smithi049.stdout:3634 punch from 0x1ac6da4 to 0x1ad62c5, (0xf521 bytes) 2024-04-24T18:08:32.913 INFO:teuthology.orchestra.run.smithi049.stdout:3635 write 0x878421e thru 0x878bc87 (0x7a6a bytes) 2024-04-24T18:08:32.915 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.916 INFO:teuthology.orchestra.run.smithi049.stdout:3637 read 0xa8f91d1 thru 0xa903c1e (0xaa4e bytes) 2024-04-24T18:08:32.916 INFO:teuthology.orchestra.run.smithi049.stdout:3640 punch from 0xb8ac16c to 0xb8b2e9a, (0x6d2e bytes) 2024-04-24T18:08:32.922 INFO:teuthology.orchestra.run.smithi049.stdout:3642 trunc from 0xdcbf72a to 0x9004364 2024-04-24T18:08:32.934 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:32.942+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:32.934 INFO:teuthology.orchestra.run.smithi049.stdout:3643 write 0x7e96339 thru 0x7ea6059 (0xfd21 bytes) 2024-04-24T18:08:32.938 INFO:teuthology.orchestra.run.smithi049.stdout:3644 write 0x65a417f thru 0x65a6afd (0x297f bytes) 2024-04-24T18:08:32.941 INFO:teuthology.orchestra.run.smithi049.stdout:3645 writesame 0x55c6b6a thru 0x55ce992 data_size 0x1(0x7e29 bytes) 2024-04-24T18:08:32.959 INFO:teuthology.orchestra.run.smithi049.stdout:3647 read 0x208a5cf thru 0x208e1ae (0x3be0 bytes) 2024-04-24T18:08:32.959 INFO:teuthology.orchestra.run.smithi049.stdout:3649 read 0x60c9ed7 thru 0x60d33c5 (0x94ef bytes) 2024-04-24T18:08:32.965 INFO:teuthology.orchestra.run.smithi049.stdout:3650 writesame 0x9b9974b thru 0x9b9ec17 data_size 0x1(0x54cd bytes) 2024-04-24T18:08:32.965 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.965 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.970 INFO:teuthology.orchestra.run.smithi049.stdout:3653 trunc from 0x9b9ec18 to 0xe93d772 2024-04-24T18:08:32.976 INFO:teuthology.orchestra.run.smithi049.stdout:3654 write 0x1c940ff thru 0x1ca15b1 (0xd4b3 bytes) 2024-04-24T18:08:32.979 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:32.979 INFO:teuthology.orchestra.run.smithi049.stdout:3656 clone 35 order 19 su 131072 sc 15 2024-04-24T18:08:33.008 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:33.008 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:33.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:33.035 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:33.043+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:33.136 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:33.144+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:33.187 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:33.187 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:33.198 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:33.212 DEBUG:teuthology.orchestra.run.smithi078:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool cache 2024-04-24T18:08:33.239 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:33.247+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:33.337 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:33.346+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:33.367 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:33.367 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:33.377 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:33.438 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:33.446+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:33.506 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:33.506 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:33.516 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:33.539 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:33.547+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:33.640 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:33.648+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:33.685 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:33.685 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:33.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:33.779 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:33.787+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:33.798 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:08:33.841 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:33.850+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:33.873 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:33.873 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:33.884 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:33.943 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:33.951+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:33.946 INFO:teuthology.orchestra.run.smithi049.stdout:flattening image image_client.0-clone34 2024-04-24T18:08:34.025 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:34.025 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:34.037 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:34.078 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:34.086+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:34.085 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:34.100 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #33, image image_client.0-clone33 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent34 2024-04-24T18:08:34.105 INFO:teuthology.orchestra.run.smithi049.stdout:3657 trunc from 0xe93d772 to 0xb790cba 2024-04-24T18:08:34.135 INFO:teuthology.orchestra.run.smithi049.stdout:3658 writesame 0x439d5ab thru 0x43a041c data_size 0xa(0x2e72 bytes) 2024-04-24T18:08:34.140 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:34.140 INFO:teuthology.orchestra.run.smithi049.stdout:3662 punch from 0x666d29b to 0x667721a, (0x9f7f bytes) 2024-04-24T18:08:34.144 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:34.153+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:34.145 INFO:teuthology.orchestra.run.smithi049.stdout:3663 write 0xba04979 thru 0xba10cd9 (0xc361 bytes) 2024-04-24T18:08:34.147 INFO:teuthology.orchestra.run.smithi049.stdout:3664 writesame 0x98e5c8e thru 0x98f4635 data_size 0x4(0xe9a8 bytes) 2024-04-24T18:08:34.156 INFO:teuthology.orchestra.run.smithi049.stdout:3665 punch from 0x49cce64 to 0x49cd2a1, (0x43d bytes) 2024-04-24T18:08:34.156 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:34.157 INFO:teuthology.orchestra.run.smithi049.stdout:3667 read 0x364a0dc thru 0x36545ea (0xa50f bytes) 2024-04-24T18:08:34.157 INFO:teuthology.orchestra.run.smithi049.stdout:3668 punch from 0x9f4122a to 0x9f4f72d, (0xe503 bytes) 2024-04-24T18:08:34.157 INFO:teuthology.orchestra.run.smithi049.stdout:3669 read 0x196a73c thru 0x1973378 (0x8c3d bytes) 2024-04-24T18:08:34.158 INFO:teuthology.orchestra.run.smithi049.stdout:3670 punch from 0xca72ff to 0xcaf04a, (0x7d4b bytes) 2024-04-24T18:08:34.158 INFO:teuthology.orchestra.run.smithi049.stdout:3671 read 0x435d52b thru 0x435f3ae (0x1e84 bytes) 2024-04-24T18:08:34.158 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:34.158 INFO:teuthology.orchestra.run.smithi049.stdout:3674 write 0x132338a thru 0x132bb7a (0x87f1 bytes) 2024-04-24T18:08:34.161 INFO:teuthology.orchestra.run.smithi049.stdout:3676 read 0xa51fe7b thru 0xa526351 (0x64d7 bytes) 2024-04-24T18:08:34.161 INFO:teuthology.orchestra.run.smithi049.stdout:3677 write 0x72d4819 thru 0x72de5d7 (0x9dbf bytes) 2024-04-24T18:08:34.166 INFO:teuthology.orchestra.run.smithi049.stdout:3678 read 0x4b13961 thru 0x4b1b224 (0x78c4 bytes) 2024-04-24T18:08:34.166 INFO:teuthology.orchestra.run.smithi049.stdout:3679 punch from 0x6cc4198 to 0x6ccb9ab, (0x7813 bytes) 2024-04-24T18:08:34.166 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:34.166 INFO:teuthology.orchestra.run.smithi049.stdout:3681 write 0x644e335 thru 0x6455f41 (0x7c0d bytes) 2024-04-24T18:08:34.167 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:34.167 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:34.169 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:34.177 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:34.181 INFO:teuthology.orchestra.run.smithi049.stdout:3683 trunc from 0xba10cda to 0x2a50234 2024-04-24T18:08:34.245 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:34.253+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:34.251 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:34.269 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:34.287 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:34.306 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:34.324 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:34.342 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:34.342 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:34.342 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:34.342 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:34.342 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:34.343 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:34.346 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:34.354+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:34.353 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:34.357 INFO:teuthology.orchestra.run.smithi078.stderr:Nothing to do for non-filestore backend 2024-04-24T18:08:34.362 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:34.362 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:34.362 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:34.362 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:34.363 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:34.363 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:34.363 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:34.363 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:34.363 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:34.365 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:34.365 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:34.399 INFO:teuthology.orchestra.run.smithi049.stdout:3684 read 0xbdb087 thru 0xbe6852 (0xb7cc bytes) 2024-04-24T18:08:34.399 INFO:teuthology.orchestra.run.smithi049.stdout:3685 read 0xafe9ef thru 0xb0549c (0x6aae bytes) 2024-04-24T18:08:34.399 INFO:teuthology.orchestra.run.smithi049.stdout:3686 read 0xe211e1 thru 0xe2d9db (0xc7fb bytes) 2024-04-24T18:08:34.400 INFO:teuthology.orchestra.run.smithi049.stdout:3687 trunc from 0x2a50234 to 0x3ad4e00 2024-04-24T18:08:34.407 INFO:teuthology.orchestra.run.smithi049.stdout:3688 trunc from 0x3ad4e00 to 0x1de2f97 2024-04-24T18:08:34.435 INFO:teuthology.orchestra.run.smithi049.stdout:3689 write 0x86ec191 thru 0x86f3f7f (0x7def bytes) 2024-04-24T18:08:34.445 INFO:teuthology.orchestra.run.smithi049.stdout:3690 writesame 0xa386f3f thru 0xa38b21e data_size 0x50(0x42e0 bytes) 2024-04-24T18:08:34.445 INFO:teuthology.orchestra.run.smithi049.stdout:3691 read 0x6477f0a thru 0x647e6b3 (0x67aa bytes) 2024-04-24T18:08:34.445 INFO:teuthology.orchestra.run.smithi049.stdout:3692 read 0x559ea5a thru 0x55a1f2f (0x34d6 bytes) 2024-04-24T18:08:34.445 INFO:teuthology.orchestra.run.smithi049.stdout:3693 writesame 0x53c18f4 thru 0x53ca8cc data_size 0x19(0x8fd9 bytes) 2024-04-24T18:08:34.447 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:34.455+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:34.447 INFO:teuthology.orchestra.run.smithi049.stdout:3695 read 0x42ac6a2 thru 0x42afd86 (0x36e5 bytes) 2024-04-24T18:08:34.457 INFO:teuthology.orchestra.run.smithi049.stdout:3696 writesame 0xe0cb0c4 thru 0xe0cfc6a data_size 0x1(0x4ba7 bytes) 2024-04-24T18:08:34.457 INFO:teuthology.orchestra.run.smithi049.stdout:3698 writesame 0x26d8168 thru 0x26db3a3 data_size 0xa(0x323c bytes) 2024-04-24T18:08:34.467 INFO:teuthology.orchestra.run.smithi049.stdout:3700 trunc from 0xe0cfc6b to 0x7e1874d 2024-04-24T18:08:34.475 INFO:teuthology.orchestra.run.smithi049.stdout:3702 read 0x533d929 thru 0x534af1b (0xd5f3 bytes) 2024-04-24T18:08:34.475 INFO:teuthology.orchestra.run.smithi049.stdout:3703 write 0x13d7cdf thru 0x13db11e (0x3440 bytes) 2024-04-24T18:08:34.477 INFO:teuthology.orchestra.run.smithi049.stdout:3704 read 0xc09d9b thru 0xc0ba0c (0x1c72 bytes) 2024-04-24T18:08:34.478 INFO:teuthology.orchestra.run.smithi049.stdout:3705 read 0x406d9a5 thru 0x4070d85 (0x33e1 bytes) 2024-04-24T18:08:34.485 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:34.486 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:34.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:34.486 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:34.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.486 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11780 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2666496~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.486 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:34.486 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.156504773999998, 2024-04-24T18:08:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042348956, 2024-04-24T18:08:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.487 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11780 2024-04-24T18:08:34.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.488 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:34.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:34.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335761+0000", 2024-04-24T18:08:34.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335768+0000", 2024-04-24T18:08:34.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1910000000000003e-06 2024-04-24T18:08:34.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335774+0000", 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9089999999999997e-06 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.335778+0000", 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9339999999999999e-06 2024-04-24T18:08:34.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378028+0000", 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042250303000000003 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378072+0000", 2024-04-24T18:08:34.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3909000000000003e-05 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.378110+0000", 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8050000000000003e-05 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.494 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.495 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11792 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2793472~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.495 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:34.495 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.154491978000003, 2024-04-24T18:08:34.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042313245999999999, 2024-04-24T18:08:34.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.496 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.496 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.496 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.496 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11792 2024-04-24T18:08:34.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.496 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:34.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337774+0000", 2024-04-24T18:08:34.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337773+0000", 2024-04-24T18:08:34.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337781+0000", 2024-04-24T18:08:34.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3089999999999998e-06 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337787+0000", 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.285e-06 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.337791+0000", 2024-04-24T18:08:34.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.065e-06 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.379933+0000", 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042141913000000003 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380016+0000", 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2959000000000003e-05 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.380087+0000", 2024-04-24T18:08:34.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1038000000000007e-05 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11799 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2856960~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.153254629999999, 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042339663999999999, 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.502 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11799 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339011+0000", 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339013+0000", 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339019+0000", 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339023+0000", 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7689999999999998e-06 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381202+0000", 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042179054000000001 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381285+0000", 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3299000000000003e-05 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381351+0000", 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5795000000000003e-05 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.153238850000001, 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.506 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:08:34.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.153222943999999, 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:08:34.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:08:34.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.153170977000002, 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.516 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11805 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2920448~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.520 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.152264277, 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042290415999999997, 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11805 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340001+0000", 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340003+0000", 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.514e-06 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340010+0000", 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340014+0000", 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.805e-06 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382196+0000", 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042182175000000002 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382236+0000", 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0265999999999997e-05 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382292+0000", 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5791000000000002e-05 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.525 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.152247502999998, 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.152230871, 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.530 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11809 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2965504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.151547418, 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042364339000000001, 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.534 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11809 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:34.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340721+0000", 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1370000000000002e-06 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340727+0000", 2024-04-24T18:08:34.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.533e-06 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340732+0000", 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5329999999999996e-06 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382924+0000", 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042192039000000001 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383012+0000", 2024-04-24T18:08:34.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7669000000000004e-05 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383083+0000", 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0988999999999994e-05 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.151529267999997, 2024-04-24T18:08:34.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.542 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.151512766000003, 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:08:34.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11814 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3012608~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:34.547 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.150618184000002, 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042354992000000001, 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11814 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341647+0000", 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341648+0000", 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.372e-06 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341652+0000", 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0690000000000002e-06 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341654+0000", 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0559999999999999e-06 2024-04-24T18:08:34.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383852+0000", 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042198125000000003 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383934+0000", 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2021999999999995e-05 2024-04-24T18:08:34.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384002+0000", 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8669000000000003e-05 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.150609899999999, 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.553 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:08:34.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.150601875, 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:34.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11819 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3065856~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.149570165999997, 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042208806000000001, 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11819 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342695+0000", 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342698+0000", 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6879999999999999e-06 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342704+0000", 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0090000000000004e-06 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342708+0000", 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9029999999999997e-06 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384745+0000", 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042037243000000002 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384833+0000", 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7769999999999995e-05 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384904+0000", 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1632999999999997e-05 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11820 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3076096~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.149553502000003, 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042328461999999997, 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.569 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11820 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:34.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342712+0000", 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342714+0000", 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342720+0000", 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8479999999999997e-06 2024-04-24T18:08:34.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342724+0000", 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6430000000000001e-06 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384944+0000", 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042220698000000001 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384992+0000", 2024-04-24T18:08:34.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7441999999999997e-05 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385041+0000", 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8715999999999999e-05 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.577 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.149537733000003, 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:34.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.149521763999999, 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.582 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:08:34.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.586 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.0893233250000005, 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:34.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:08:34.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:08:34.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:34.592 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:34.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:34.595 INFO:teuthology.orchestra.run.smithi049.stdout:3706 write 0x9ef6cd0 thru 0x9f04694 (0xd9c5 bytes) 2024-04-24T18:08:34.595 INFO:teuthology.orchestra.run.smithi049.stdout:3707 punch from 0x6506132 to 0x6510d00, (0xabce bytes) 2024-04-24T18:08:34.595 INFO:teuthology.orchestra.run.smithi049.stdout:3708 write 0x66f00d thru 0x67cc14 (0xdc08 bytes) 2024-04-24T18:08:34.595 INFO:teuthology.orchestra.run.smithi049.stdout:3709 trunc from 0x9f04695 to 0x92feaae 2024-04-24T18:08:34.595 INFO:teuthology.orchestra.run.smithi049.stdout:3710 write 0xb8b031a thru 0xb8b4610 (0x42f7 bytes) 2024-04-24T18:08:34.596 INFO:teuthology.orchestra.run.smithi049.stdout:3712 writesame 0x6fe237b thru 0x6fe5701 data_size 0x1(0x3387 bytes) 2024-04-24T18:08:34.596 INFO:teuthology.orchestra.run.smithi049.stdout:3714 write 0x8fb4b5c thru 0x8fc4113 (0xf5b8 bytes) 2024-04-24T18:08:34.596 INFO:teuthology.orchestra.run.smithi049.stdout:3715 writesame 0x1962ca thru 0x1a18a6 data_size 0x15(0xb5dd bytes) 2024-04-24T18:08:34.596 INFO:teuthology.orchestra.run.smithi049.stdout:3716 read 0x5f3b79e thru 0x5f4598c (0xa1ef bytes) 2024-04-24T18:08:34.596 INFO:teuthology.orchestra.run.smithi049.stdout:3717 write 0xd4bfc02 thru 0xd4c994b (0x9d4a bytes) 2024-04-24T18:08:34.596 INFO:teuthology.orchestra.run.smithi049.stdout:3718 write 0x1499a8f thru 0x14a88d2 (0xee44 bytes) 2024-04-24T18:08:34.596 INFO:teuthology.orchestra.run.smithi049.stdout:3720 clone 36 order 18 su 1024 sc 7 2024-04-24T18:08:34.596 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:34.601+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:34.695 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:34.703+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:34.760 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:34.760 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:34.760 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:34.760 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:34.770 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:34.795 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:34.803+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:34.856 INFO:teuthology.orchestra.run.smithi049.stdout:flattening image image_client.0-clone35 2024-04-24T18:08:34.887 DEBUG:teuthology.orchestra.run.smithi078:> CEPH_ARGS='--filestore-merge-threshold 1 --filestore-split-multiple 1' sudo -E ceph-objectstore-tool --no-mon-config --log-file=/var/log/ceph/objectstore_tool.$pid.log --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op apply-layout-settings --pool pool_client.0 2024-04-24T18:08:34.896 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:34.904+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:34.939 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:34.940 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:34.940 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:34.940 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:34.940 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:34.951 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:34.997 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:35.005+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:35.014 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #34, image image_client.0-clone34 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent35 2024-04-24T18:08:35.076 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:35.076 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:35.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:35.076 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:35.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.076 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.076 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7196559799999998, 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:08:35.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:08:35.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.080 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7196478509999995, 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:35.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:08:35.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.084 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.719639828, 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.088 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7196309149999998, 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.092 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7196220960000002, 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7196129639999995, 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:08:35.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.100 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7187927050000003, 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:08:35.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:08:35.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.104 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7187655060000004, 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:08:35.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:08:35.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.108 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.718748036, 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:35.112 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.718732696, 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:35.116 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.718708618, 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.718693053, 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:08:35.120 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:08:35.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7186780920000002, 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.124 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7186611530000002, 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.128 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:08:35.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.718644748, 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:08:35.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:08:35.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:08:35.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.135 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7186283089999996, 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7186134690000001, 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:08:35.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.143 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7185970450000001, 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:08:35.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:08:35.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.147 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7185798300000004, 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:08:35.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.7185623650000004, 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:08:35.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:35.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:08:35.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.155 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:35.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:35.157 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:35.164+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:35.257 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:35.265+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:35.273 INFO:teuthology.orchestra.run.smithi049.stdout:3722 punch from 0x5346a9 to 0x5351e1, (0xb38 bytes) 2024-04-24T18:08:35.277 INFO:teuthology.orchestra.run.smithi049.stdout:3724 write 0x5ed86e0 thru 0x5edb383 (0x2ca4 bytes) 2024-04-24T18:08:35.287 INFO:teuthology.orchestra.run.smithi049.stdout:3725 write 0x37ffd99 thru 0x3801ec9 (0x2131 bytes) 2024-04-24T18:08:35.291 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:35.291 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:35.292 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:35.292 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:35.298 INFO:teuthology.orchestra.run.smithi049.stdout:3726 write 0x6923ff1 thru 0x692de07 (0x9e17 bytes) 2024-04-24T18:08:35.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:35.322 INFO:teuthology.orchestra.run.smithi049.stdout:3727 punch from 0x9bd4bf7 to 0x9bdc735, (0x7b3e bytes) 2024-04-24T18:08:35.322 INFO:teuthology.orchestra.run.smithi049.stdout:3728 read 0xa1b6a6f thru 0xa1c5ecb (0xf45d bytes) 2024-04-24T18:08:35.322 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.322 INFO:teuthology.orchestra.run.smithi049.stdout:3731 write 0x2ad34e1 thru 0x2adc7de (0x92fe bytes) 2024-04-24T18:08:35.341 INFO:teuthology.orchestra.run.smithi049.stdout:3732 read 0xc56de29 thru 0xc57133e (0x3516 bytes) 2024-04-24T18:08:35.341 INFO:teuthology.orchestra.run.smithi049.stdout:3733 write 0x10a1d0f thru 0x10a9a01 (0x7cf3 bytes) 2024-04-24T18:08:35.358 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:35.365+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:35.376 INFO:teuthology.orchestra.run.smithi049.stdout:3734 writesame 0xa15d1af thru 0xa164559 data_size 0x1(0x73ab bytes) 2024-04-24T18:08:35.388 INFO:teuthology.orchestra.run.smithi049.stdout:3735 write 0x980a736 thru 0x9814fc1 (0xa88c bytes) 2024-04-24T18:08:35.398 INFO:teuthology.orchestra.run.smithi049.stdout:3736 writesame 0x450e513 thru 0x45104af data_size 0x1(0x1f9d bytes) 2024-04-24T18:08:35.408 INFO:teuthology.orchestra.run.smithi049.stdout:3737 punch from 0xa5d013 to 0xa699e9, (0xc9d6 bytes) 2024-04-24T18:08:35.408 INFO:teuthology.orchestra.run.smithi049.stdout:3738 write 0x34a385b thru 0x34aacb3 (0x7459 bytes) 2024-04-24T18:08:35.418 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.418 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.418 INFO:teuthology.orchestra.run.smithi049.stdout:3742 punch from 0x5567a96 to 0x5574bd6, (0xd140 bytes) 2024-04-24T18:08:35.418 INFO:teuthology.orchestra.run.smithi049.stdout:3744 writesame 0x5f2cfe6 thru 0x5f2d843 data_size 0x1(0x85e bytes) 2024-04-24T18:08:35.420 INFO:teuthology.orchestra.run.smithi049.stdout:3745 writesame 0x7e6ed7e thru 0x7e72c20 data_size 0x5(0x3ea3 bytes) 2024-04-24T18:08:35.430 INFO:teuthology.orchestra.run.smithi049.stdout:3746 read 0x4209c33 thru 0x420c345 (0x2713 bytes) 2024-04-24T18:08:35.430 INFO:teuthology.orchestra.run.smithi049.stdout:3747 write 0x1b82056 thru 0x1b8c2c3 (0xa26e bytes) 2024-04-24T18:08:35.459 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:35.466+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:35.461 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.467 INFO:teuthology.orchestra.run.smithi049.stdout:3749 write 0xd592c77 thru 0xd5939f8 (0xd82 bytes) 2024-04-24T18:08:35.473 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.473 INFO:teuthology.orchestra.run.smithi049.stdout:3751 punch from 0xb607d26 to 0xb60d6a9, (0x5983 bytes) 2024-04-24T18:08:35.473 INFO:teuthology.orchestra.run.smithi049.stdout:3754 write 0xa5ac73e thru 0xa5bb38e (0xec51 bytes) 2024-04-24T18:08:35.480 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:35.480 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:35.480 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:35.480 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:35.480 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:35.491 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:35.494 INFO:teuthology.orchestra.run.smithi049.stdout:3755 trunc from 0xd5939f9 to 0x4830cd4 2024-04-24T18:08:35.529 INFO:teuthology.orchestra.run.smithi049.stdout:3757 write 0x63c7971 thru 0x63d4dd5 (0xd465 bytes) 2024-04-24T18:08:35.545 INFO:teuthology.orchestra.run.smithi049.stdout:3758 write 0x8bc8617 thru 0x8bd3a23 (0xb40d bytes) 2024-04-24T18:08:35.554 INFO:teuthology.orchestra.run.smithi049.stdout:3759 punch from 0x72046ec to 0x720d8e4, (0x91f8 bytes) 2024-04-24T18:08:35.554 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.557 INFO:teuthology.orchestra.run.smithi049.stdout:3762 trunc from 0x8bd3a24 to 0x5b834ea 2024-04-24T18:08:35.559 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:35.567+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:35.568 INFO:teuthology.orchestra.run.smithi049.stdout:3764 trunc from 0x5b834ea to 0x56b788b 2024-04-24T18:08:35.573 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.583 INFO:teuthology.orchestra.run.smithi049.stdout:3769 write 0x9f8a2d7 thru 0x9f8a402 (0x12c bytes) 2024-04-24T18:08:35.586 INFO:teuthology.orchestra.run.smithi049.stdout:3770 punch from 0x43073d5 to 0x4311967, (0xa592 bytes) 2024-04-24T18:08:35.587 INFO:teuthology.orchestra.run.smithi049.stdout:3771 read 0x48b259f thru 0x48b9224 (0x6c86 bytes) 2024-04-24T18:08:35.587 INFO:teuthology.orchestra.run.smithi049.stdout:3772 read 0x2ef04e5 thru 0x2efa861 (0xa37d bytes) 2024-04-24T18:08:35.587 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:35.587 INFO:teuthology.orchestra.run.smithi049.stdout:3775 write 0x61b2031 thru 0x61b9a03 (0x79d3 bytes) 2024-04-24T18:08:35.596 INFO:teuthology.orchestra.run.smithi049.stdout:3776 read 0x708bdc1 thru 0x7095aa1 (0x9ce1 bytes) 2024-04-24T18:08:35.605 INFO:teuthology.orchestra.run.smithi049.stdout:3779 write 0xd929e6e thru 0xd936772 (0xc905 bytes) 2024-04-24T18:08:35.613 INFO:teuthology.orchestra.run.smithi049.stdout:3780 write 0xa58f2fb thru 0xa5931f9 (0x3eff bytes) 2024-04-24T18:08:35.621 INFO:teuthology.orchestra.run.smithi049.stdout:3781 read 0x615f88c thru 0x616cee2 (0xd657 bytes) 2024-04-24T18:08:35.622 INFO:teuthology.orchestra.run.smithi049.stdout:3782 write 0x18fcb1a thru 0x1901e53 (0x533a bytes) 2024-04-24T18:08:35.629 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020507615, 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.630 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.632 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.635 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020438562999999, 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.636 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.638 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020414148, 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.641 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:35.646 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020390999, 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.647 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.648 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.649 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020310430999999, 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.652 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.657 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020285620999999, 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020265508999998, 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020194845999995, 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.668 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:35.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:08:35.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:08:35.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:08:35.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.020183072999998, 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.673 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:35.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:08:35.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.678 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.599861243999996, 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.599814061999993, 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.599788941, 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.689 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:08:35.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:08:35.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:08:35.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.599736806999999, 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:08:35.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:08:35.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:08:35.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:08:35.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.699 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.599673343000006, 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:35.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:08:35.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:08:35.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.704 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.088467671000004, 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:08:35.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:08:35.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:08:35.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:08:35.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:08:35.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.088442643999997, 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:35.715 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.088363407000003, 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:35.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.826174625999997, 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:35.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:08:35.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:08:35.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:08:35.724 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.825068557000002, 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:08:35.728 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 3.6698968389999997, 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:35.732 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:35.733 INFO:teuthology.orchestra.run.smithi049.stdout:3783 clone 37 order 23 su 65536 sc 9 2024-04-24T18:08:35.734 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:35.735 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:35.741+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:35.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:08:35.855 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:35.863+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:35.858 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone36 from 0xd936773 (overlap 0x4830cd4) to 0x40f8b8b 2024-04-24T18:08:35.889 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:35.889 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:35.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.889 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130569 4.b 4:d001b633:::rbd_data.11745ecef76e.000000000000027f:head [delete] snapc 28=[28] ondisk+write+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:35.889 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:35.889723+0000", 2024-04-24T18:08:35.889 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.0031488589999999999, 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0031775449999999999, 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "waiting for sub ops", 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130569 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.889723+0000", 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:35.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.889723+0000", 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.889722+0000", 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.889729+0000", 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7560000000000002e-06 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.889741+0000", 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2714000000000001e-05 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.889748+0000", 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8120000000000002e-06 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.890576+0000", 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00082806799999999995 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.890650+0000", 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3768000000000003e-05 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:35.890814+0000", 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00016443799999999999 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 1 2024-04-24T18:08:35.893 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:35.898 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:35.940 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:35.948+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:35.983 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #35, image image_client.0-clone35 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent36 2024-04-24T18:08:35.984 INFO:teuthology.orchestra.run.smithi078.stderr:Nothing to do for non-filestore backend 2024-04-24T18:08:36.037 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:36.045+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:36.097 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:36.097 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:36.097 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:36.109 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:36.138 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:36.146+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:36.239 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:36.247+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:36.288 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:36.288 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:36.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:36.288 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:36.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.589150283999999, 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 39.676675478, 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:36.294 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.299 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24920695500000001, 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24918567, 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:08:36.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24916830700000001, 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.249122914, 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.310 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.313 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24909541299999999, 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:36.317 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24907685299999999, 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.320 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24905296499999999, 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:08:36.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.249035071, 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.249000429, 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:36.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24890590100000001, 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24888721999999999, 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.248831741, 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.248797355, 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24877539600000001, 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24875831300000001, 2024-04-24T18:08:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:08:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24874185300000001, 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:08:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:08:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24866793800000001, 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:08:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:08:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 0.24861541500000001, 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:08:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:36.364 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:36.365 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:36.366 INFO:teuthology.orchestra.run.smithi049.stdout:3784 trunc from 0xd936773 to 0xb797129 2024-04-24T18:08:36.366 INFO:teuthology.orchestra.run.smithi049.stdout:3785 read 0x29b4643 thru 0x29b889f (0x425d bytes) 2024-04-24T18:08:36.366 INFO:teuthology.orchestra.run.smithi049.stdout:3786 trunc from 0xb797129 to 0x33e3645 2024-04-24T18:08:36.366 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:36.367 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:36.373+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:36.466 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:36.474+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:36.569 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:36.577+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:36.633 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:36.633 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:36.633 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:36.633 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:36.645 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:36.653 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:36.668 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:36.676+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:36.671 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:36.693 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:36.712 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:36.730 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:36.749 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:36.768 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:36.777+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:36.785 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:36.785 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:36.786 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:36.789 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:36.789 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:36.798 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:36.798 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:36.813 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:36.813 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:08:36.907 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:36.915+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:36.970 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:36.977+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:36.997 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:36.997 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:37.008 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:37.071 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:37.079+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:37.172 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:37.180+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:37.214 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:37.215 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:37.215 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:37.299 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:37.307+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:37.373 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:37.381+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:37.389 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:37.389 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:37.405 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:37.492 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:37.500+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:37.550 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.550 INFO:teuthology.orchestra.run.smithi049.stdout:3788 read 0x1a416c1 thru 0x1a4c122 (0xaa62 bytes) 2024-04-24T18:08:37.559 INFO:teuthology.orchestra.run.smithi049.stdout:3789 write 0x5d3660a thru 0x5d3c0bd (0x5ab4 bytes) 2024-04-24T18:08:37.575 INFO:teuthology.orchestra.run.smithi049.stdout:3790 write 0xb7a07e4 thru 0xb7a663d (0x5e5a bytes) 2024-04-24T18:08:37.575 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:37.583+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:37.578 INFO:teuthology.orchestra.run.smithi049.stdout:3791 punch from 0x4809c1f to 0x480fae6, (0x5ec7 bytes) 2024-04-24T18:08:37.578 INFO:teuthology.orchestra.run.smithi049.stdout:3792 writesame 0x70b6acf thru 0x70b82ad data_size 0x15(0x17df bytes) 2024-04-24T18:08:37.581 INFO:teuthology.orchestra.run.smithi049.stdout:3793 trunc from 0xb7a663e to 0xb831f40 2024-04-24T18:08:37.594 INFO:teuthology.orchestra.run.smithi049.stdout:3794 write 0xe3a31b9 thru 0xe3adf91 (0xadd9 bytes) 2024-04-24T18:08:37.597 INFO:teuthology.orchestra.run.smithi049.stdout:3796 read 0x18a1a1c thru 0x18adaeb (0xc0d0 bytes) 2024-04-24T18:08:37.597 INFO:teuthology.orchestra.run.smithi049.stdout:3797 write 0x18fe74d thru 0x19002d9 (0x1b8d bytes) 2024-04-24T18:08:37.599 INFO:teuthology.orchestra.run.smithi049.stdout:3798 write 0x3859197 thru 0x3862a67 (0x98d1 bytes) 2024-04-24T18:08:37.607 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:37.607 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:37.611 INFO:teuthology.orchestra.run.smithi049.stdout:3800 trunc from 0xe3adf92 to 0x4db781e 2024-04-24T18:08:37.621 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:37.622 INFO:teuthology.orchestra.run.smithi049.stdout:3802 read 0x346ecd6 thru 0x3474e08 (0x6133 bytes) 2024-04-24T18:08:37.629 INFO:teuthology.orchestra.run.smithi049.stdout:3803 writesame 0x5f5b143 thru 0x5f64a39 data_size 0x13(0x98f7 bytes) 2024-04-24T18:08:37.629 INFO:teuthology.orchestra.run.smithi049.stdout:3804 trunc from 0x5f64a3a to 0x5acaebe 2024-04-24T18:08:37.636 INFO:teuthology.orchestra.run.smithi049.stdout:3806 read 0x57c67f9 thru 0x57cdcda (0x74e2 bytes) 2024-04-24T18:08:37.636 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.636 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.636 INFO:teuthology.orchestra.run.smithi049.stdout:3810 read 0x5779b18 thru 0x57858f6 (0xbddf bytes) 2024-04-24T18:08:37.636 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.649 INFO:teuthology.orchestra.run.smithi049.stdout:3813 write 0xc1f955f thru 0xc1f95bb (0x5d bytes) 2024-04-24T18:08:37.652 INFO:teuthology.orchestra.run.smithi049.stdout:3814 punch from 0x31987 to 0x3e340, (0xc9b9 bytes) 2024-04-24T18:08:37.652 INFO:teuthology.orchestra.run.smithi049.stdout:3815 punch from 0xb90bec9 to 0xb91586f, (0x99a6 bytes) 2024-04-24T18:08:37.658 INFO:teuthology.orchestra.run.smithi049.stdout:3816 trunc from 0xc1f95bc to 0x6e79429 2024-04-24T18:08:37.668 INFO:teuthology.orchestra.run.smithi049.stdout:3817 write 0x342739b thru 0x34350b6 (0xdd1c bytes) 2024-04-24T18:08:37.675 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:37.683+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:37.681 INFO:teuthology.orchestra.run.smithi049.stdout:3818 writesame 0xa6515d1 thru 0xa656100 data_size 0x18(0x4b30 bytes) 2024-04-24T18:08:37.681 INFO:teuthology.orchestra.run.smithi049.stdout:3819 write 0x21e44d7 thru 0x21f421e (0xfd48 bytes) 2024-04-24T18:08:37.683 INFO:teuthology.orchestra.run.smithi049.stdout:3820 writesame 0x5c0c538 thru 0x5c1bcf3 data_size 0xf(0xf7bc bytes) 2024-04-24T18:08:37.688 INFO:teuthology.orchestra.run.smithi049.stdout:3824 trunc from 0xa656101 to 0x95f8bda 2024-04-24T18:08:37.693 INFO:teuthology.orchestra.run.smithi049.stdout:3825 write 0x1c0c37a thru 0x1c14842 (0x84c9 bytes) 2024-04-24T18:08:37.695 INFO:teuthology.orchestra.run.smithi049.stdout:3826 write 0x33ee8b4 thru 0x33f6630 (0x7d7d bytes) 2024-04-24T18:08:37.706 INFO:teuthology.orchestra.run.smithi049.stdout:3827 writesame 0xce8071f thru 0xce8ebb3 data_size 0x1(0xe495 bytes) 2024-04-24T18:08:37.706 INFO:teuthology.orchestra.run.smithi049.stdout:3828 write 0x82d6fa4 thru 0x82d90e7 (0x2144 bytes) 2024-04-24T18:08:37.708 INFO:teuthology.orchestra.run.smithi049.stdout:3829 writesame 0x48a675b thru 0x48a85a1 data_size 0x1(0x1e47 bytes) 2024-04-24T18:08:37.713 INFO:teuthology.orchestra.run.smithi049.stdout:3830 read 0x43b0605 thru 0x43b6e95 (0x6891 bytes) 2024-04-24T18:08:37.714 INFO:teuthology.orchestra.run.smithi049.stdout:3831 punch from 0x2ca775a to 0x2cad8ed, (0x6193 bytes) 2024-04-24T18:08:37.714 INFO:teuthology.orchestra.run.smithi049.stdout:3832 read 0x21900e7 thru 0x219cccf (0xcbe9 bytes) 2024-04-24T18:08:37.722 INFO:teuthology.orchestra.run.smithi049.stdout:3834 writesame 0xe916341 thru 0xe91dade data_size 0x3d(0x779e bytes) 2024-04-24T18:08:37.722 INFO:teuthology.orchestra.run.smithi049.stdout:3837 punch from 0x6b5ce7c to 0x6b63704, (0x6888 bytes) 2024-04-24T18:08:37.722 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.722 INFO:teuthology.orchestra.run.smithi049.stdout:3840 write 0x44487c7 thru 0x4456d7c (0xe5b6 bytes) 2024-04-24T18:08:37.739 INFO:teuthology.orchestra.run.smithi049.stdout:3841 trunc from 0xe91dadf to 0x2aa63a 2024-04-24T18:08:37.764 INFO:teuthology.orchestra.run.smithi049.stdout:3842 writesame 0x2831f04 thru 0x28373e6 data_size 0x1(0x54e3 bytes) 2024-04-24T18:08:37.770 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:37.770 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:37.776 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:37.784+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:37.783 INFO:teuthology.orchestra.run.smithi049.stdout:3843 write 0xdd4dd5a thru 0xdd559de (0x7c85 bytes) 2024-04-24T18:08:37.786 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:37.787 INFO:teuthology.orchestra.run.smithi049.stdout:3846 write 0xbe68229 thru 0xbe69ce4 (0x1abc bytes) 2024-04-24T18:08:37.789 INFO:teuthology.orchestra.run.smithi049.stdout:3847 writesame 0x61d2540 thru 0x61d2e1e data_size 0x1(0x8df bytes) 2024-04-24T18:08:37.792 INFO:teuthology.orchestra.run.smithi049.stdout:3848 read 0xce99494 thru 0xcea8993 (0xf500 bytes) 2024-04-24T18:08:37.792 INFO:teuthology.orchestra.run.smithi049.stdout:3849 read 0x3b686ca thru 0x3b71f6a (0x98a1 bytes) 2024-04-24T18:08:37.793 INFO:teuthology.orchestra.run.smithi049.stdout:3850 read 0x5abe99a thru 0x5acbe93 (0xd4fa bytes) 2024-04-24T18:08:37.793 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.793 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.793 INFO:teuthology.orchestra.run.smithi049.stdout:3853 writesame 0x6af3cb7 thru 0x6b01293 data_size 0x1(0xd5dd bytes) 2024-04-24T18:08:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:3854 write 0x589f2b thru 0x593d64 (0x9e3a bytes) 2024-04-24T18:08:37.825 INFO:teuthology.orchestra.run.smithi049.stdout:3856 punch from 0xbb35bb2 to 0xbb44fdc, (0xf42a bytes) 2024-04-24T18:08:37.825 INFO:teuthology.orchestra.run.smithi049.stdout:3858 write 0xc198304 thru 0xc19ac18 (0x2915 bytes) 2024-04-24T18:08:37.828 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:37.828 INFO:teuthology.orchestra.run.smithi049.stdout:3860 writesame 0xb8c2afc thru 0xb8cf905 data_size 0x95(0xce0a bytes) 2024-04-24T18:08:37.829 INFO:teuthology.orchestra.run.smithi049.stdout:3861 read 0xb0f2769 thru 0xb0f6bd4 (0x446c bytes) 2024-04-24T18:08:37.842 INFO:teuthology.orchestra.run.smithi049.stdout:3862 trunc from 0xdd559df to 0x2f304e1 2024-04-24T18:08:37.855 INFO:teuthology.orchestra.run.smithi049.stdout:3864 clone 38 order 23 su 4194304 sc 5 2024-04-24T18:08:37.877 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:37.885+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:37.925 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:37.925 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:37.935 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:37.978 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:37.985+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:38.079 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:38.087+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:38.105 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:38.105 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:38.115 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:38.179 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:38.187+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:38.241 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:38.241 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:38.251 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:38.280 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:38.289+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:38.381 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:38.390+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:38.414 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:38.415 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:38.424 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:38.483 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:38.491+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:38.548 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:38.548 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:38.558 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:38.584 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:38.591+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:38.684 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:38.692+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:38.723 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:38.723 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:38.733 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:38.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:38.816 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:38.825+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:38.825 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:08:38.879 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone37 from 0x2f304e1 (overlap 0x2aa63a) to 0x0 2024-04-24T18:08:38.886 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:38.894+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:38.904 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:38.904 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:38.915 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:38.946 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #36, image image_client.0-clone36 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent37 2024-04-24T18:08:38.968 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:38.992 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:39.003 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:39.009+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:39.011 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:39.031 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:39.065 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:39.088 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:39.096+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:39.099 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7768612350000001, 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.134 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:39.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:08:39.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:08:39.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7768531060000008, 2024-04-24T18:08:39.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.142 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7768450829999995, 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:39.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:08:39.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:08:39.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.146 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7768361699999993, 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.150 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7768273509999997, 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:08:39.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:08:39.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:08:39.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:08:39.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7768182190000008, 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.155 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:08:39.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:08:39.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7759979599999998, 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.159 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.775970761, 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7759532910000004, 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.167 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7759379509999995, 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.171 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:08:39.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7759138730000004, 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:39.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:08:39.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:08:39.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7758983080000004, 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.179 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7758833470000006, 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.183 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7758664080000006, 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.187 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:08:39.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7758500030000004, 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.191 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:39.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:08:39.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:08:39.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7758335639999991, 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.195 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:39.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:08:39.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.199 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7758187240000005, 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:08:39.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:08:39.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7758023000000005, 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7757850850000008, 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.7757676199999999, 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.215 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.216 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.216 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.216 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.216 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:39.217 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:39.217 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:39.218 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:39.218 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:39.225+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:39.219 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:39.219 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:39.318 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:39.325+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:39.391 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:39.391 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:39.391 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:39.401 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:39.419 INFO:teuthology.orchestra.run.smithi049.stdout:3865 write 0x8418087 thru 0x841fab3 (0x7a2d bytes) 2024-04-24T18:08:39.422 INFO:teuthology.orchestra.run.smithi049.stdout:3866 write 0x798027d thru 0x798dd73 (0xdaf7 bytes) 2024-04-24T18:08:39.434 INFO:teuthology.orchestra.run.smithi049.stdout:3868 write 0xc6234e6 thru 0xc632aa6 (0xf5c1 bytes) 2024-04-24T18:08:39.438 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.438 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.438 INFO:teuthology.orchestra.run.smithi049.stdout:3871 write 0x602f14f thru 0x6038721 (0x95d3 bytes) 2024-04-24T18:08:39.441 INFO:teuthology.orchestra.run.smithi049.stdout:3872 punch from 0xb01bb57 to 0xb029f83, (0xe42c bytes) 2024-04-24T18:08:39.441 INFO:teuthology.orchestra.run.smithi049.stdout:3873 write 0x10cb1f1 thru 0x10d14c1 (0x62d1 bytes) 2024-04-24T18:08:39.445 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:39.453+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:39.445 INFO:teuthology.orchestra.run.smithi049.stdout:3875 read 0xbaca48a thru 0xbad117e (0x6cf5 bytes) 2024-04-24T18:08:39.445 INFO:teuthology.orchestra.run.smithi049.stdout:3877 writesame 0xa3832a4 thru 0xa38c462 data_size 0x1(0x91bf bytes) 2024-04-24T18:08:39.459 INFO:teuthology.orchestra.run.smithi049.stdout:3878 read 0x237b24d thru 0x23878b0 (0xc664 bytes) 2024-04-24T18:08:39.460 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.460 INFO:teuthology.orchestra.run.smithi049.stdout:3880 read 0x55c5f48 thru 0x55cf237 (0x92f0 bytes) 2024-04-24T18:08:39.460 INFO:teuthology.orchestra.run.smithi049.stdout:3881 write 0x252e268 thru 0x2533f54 (0x5ced bytes) 2024-04-24T18:08:39.462 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.462 INFO:teuthology.orchestra.run.smithi049.stdout:3883 read 0x8722228 thru 0x872b73a (0x9513 bytes) 2024-04-24T18:08:39.463 INFO:teuthology.orchestra.run.smithi049.stdout:3884 write 0x5dd8ce4 thru 0x5dd8f2a (0x247 bytes) 2024-04-24T18:08:39.470 INFO:teuthology.orchestra.run.smithi049.stdout:3886 trunc from 0xc632aa7 to 0x8134c17 2024-04-24T18:08:39.493 INFO:teuthology.orchestra.run.smithi049.stdout:3887 writesame 0xc99f68f thru 0xc9a904c data_size 0xb(0x99be bytes) 2024-04-24T18:08:39.493 INFO:teuthology.orchestra.run.smithi049.stdout:3888 write 0x96b5346 thru 0x96bc414 (0x70cf bytes) 2024-04-24T18:08:39.496 INFO:teuthology.orchestra.run.smithi049.stdout:3891 punch from 0x143f8fc to 0x14448d9, (0x4fdd bytes) 2024-04-24T18:08:39.503 INFO:teuthology.orchestra.run.smithi049.stdout:3892 write 0xe871969 thru 0xe880ef8 (0xf590 bytes) 2024-04-24T18:08:39.506 INFO:teuthology.orchestra.run.smithi049.stdout:3893 write 0x4386eb4 thru 0x43907b3 (0x9900 bytes) 2024-04-24T18:08:39.510 INFO:teuthology.orchestra.run.smithi049.stdout:3894 read 0x6902251 thru 0x690dc77 (0xba27 bytes) 2024-04-24T18:08:39.517 INFO:teuthology.orchestra.run.smithi049.stdout:3895 trunc from 0xe880ef9 to 0x8657abd 2024-04-24T18:08:39.519 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:39.527+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:39.528 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.535 INFO:teuthology.orchestra.run.smithi049.stdout:3897 writesame 0x9dc7d85 thru 0x9dd1807 data_size 0xf(0x9a83 bytes) 2024-04-24T18:08:39.537 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:39.537 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:39.537 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:39.538 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:39.538 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:39.542 INFO:teuthology.orchestra.run.smithi049.stdout:3898 writesame 0xb01dba5 thru 0xb0292af data_size 0x1(0xb70b bytes) 2024-04-24T18:08:39.542 INFO:teuthology.orchestra.run.smithi049.stdout:3899 write 0x96cd416 thru 0x96cd711 (0x2fc bytes) 2024-04-24T18:08:39.544 INFO:teuthology.orchestra.run.smithi049.stdout:3900 punch from 0xaffc261 to 0xb003e49, (0x7be8 bytes) 2024-04-24T18:08:39.544 INFO:teuthology.orchestra.run.smithi049.stdout:3901 punch from 0x4ff90ef to 0x4ffbaf7, (0x2a08 bytes) 2024-04-24T18:08:39.544 INFO:teuthology.orchestra.run.smithi049.stdout:3902 punch from 0xf50d11 to 0xf5fdd2, (0xf0c1 bytes) 2024-04-24T18:08:39.544 INFO:teuthology.orchestra.run.smithi049.stdout:3903 read 0x9e56fa7 thru 0x9e57804 (0x85e bytes) 2024-04-24T18:08:39.546 INFO:teuthology.orchestra.run.smithi049.stdout:3904 trunc from 0xb0292b0 to 0x86fd3bb 2024-04-24T18:08:39.548 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:39.553 INFO:teuthology.orchestra.run.smithi049.stdout:3905 read 0x4e4057c thru 0x4e501e9 (0xfc6e bytes) 2024-04-24T18:08:39.553 INFO:teuthology.orchestra.run.smithi049.stdout:3906 writesame 0x2dd7a9a thru 0x2dd9fcd data_size 0x2(0x2534 bytes) 2024-04-24T18:08:39.564 INFO:teuthology.orchestra.run.smithi049.stdout:3908 write 0xad6693d thru 0xad6d3e3 (0x6aa7 bytes) 2024-04-24T18:08:39.567 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.567 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.567 INFO:teuthology.orchestra.run.smithi049.stdout:3911 read 0x5024b91 thru 0x5029edf (0x534f bytes) 2024-04-24T18:08:39.567 INFO:teuthology.orchestra.run.smithi049.stdout:3913 punch from 0x947314a to 0x947d72f, (0xa5e5 bytes) 2024-04-24T18:08:39.567 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.568 INFO:teuthology.orchestra.run.smithi049.stdout:3916 write 0x5e1f432 thru 0x5e27502 (0x80d1 bytes) 2024-04-24T18:08:39.568 INFO:teuthology.orchestra.run.smithi049.stdout:3917 read 0x13783ea thru 0x1381c9c (0x98b3 bytes) 2024-04-24T18:08:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:3919 punch from 0x12a1020 to 0x12ab8a3, (0xa883 bytes) 2024-04-24T18:08:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.569 INFO:teuthology.orchestra.run.smithi049.stdout:3923 writesame 0x6f57b90 thru 0x6f5ef09 data_size 0x1a(0x737a bytes) 2024-04-24T18:08:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:3924 writesame 0xe439ecc thru 0xe448cb0 data_size 0x1(0xede5 bytes) 2024-04-24T18:08:39.581 INFO:teuthology.orchestra.run.smithi049.stdout:3925 write 0x200db0d thru 0x2014cf8 (0x71ec bytes) 2024-04-24T18:08:39.587 INFO:teuthology.orchestra.run.smithi049.stdout:3926 trunc from 0xe448cb1 to 0xdad1a48 2024-04-24T18:08:39.592 INFO:teuthology.orchestra.run.smithi049.stdout:3927 read 0x1d9fa66 thru 0x1dacab1 (0xd04c bytes) 2024-04-24T18:08:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:3930 read 0x5bfe8fa thru 0x5c0d9fd (0xf104 bytes) 2024-04-24T18:08:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:39.593 INFO:teuthology.orchestra.run.smithi049.stdout:3932 clone 39 order 22 su 2097152 sc 11 2024-04-24T18:08:39.620 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:39.627+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104809191000001, 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:08:39.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104740139, 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:08:39.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:08:39.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104715724000002, 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:08:39.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:39.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104692575000001, 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:39.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104612007, 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.735 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:08:39.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:08:39.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:08:39.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:08:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.741 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104587197000001, 2024-04-24T18:08:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:08:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.742 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:08:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:08:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:08:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:08:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.746 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:08:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:08:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:08:39.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:08:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:08:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:08:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:08:39.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:08:39.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:08:39.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:08:39.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:08:39.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:08:39.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:08:39.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:08:39.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.753 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.753 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.754 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:39.754 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104567084999999, 2024-04-24T18:08:39.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:08:39.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.754 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:08:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:39.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:39.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:39.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:08:39.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:08:39.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:08:39.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:08:39.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:08:39.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:08:39.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:08:39.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:08:39.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:08:39.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:08:39.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.762 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:08:39.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:08:39.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.762 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.762 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:08:39.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:08:39.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:08:39.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:08:39.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.765 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104496421999997, 2024-04-24T18:08:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:08:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.766 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:08:39.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.767 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.767 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:39.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:08:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:08:39.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:08:39.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:39.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.773 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.775 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.104484649, 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:08:39.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:08:39.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.684162819999997, 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.781 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.684115637999994, 2024-04-24T18:08:39.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:39.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.684090517000001, 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.791 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.793 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:39.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.793 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:39.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:08:39.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.684038383000001, 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.798 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:08:39.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.683974918999994, 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.803 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:08:39.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:08:39.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:08:39.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:08:39.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:08:39.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.809 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.809 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:08:39.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:08:39.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:08:39.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:08:39.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:08:39.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:08:39.811 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.172769247000005, 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:08:39.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.813 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.813 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.813 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.813 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:08:39.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:08:39.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:08:39.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.172744219999998, 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.819 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:08:39.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.824 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.172664983000004, 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:08:39.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.829 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.910476201999998, 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:08:39.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.909370133000003, 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:08:39.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:08:39.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 7.7541984150000003, 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:08:39.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:08:39.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.841 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:39.842 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:39.842 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:39.843 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:08:39.844 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:39.851+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:39.844 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:39.886 INFO:teuthology.orchestra.run.smithi049.stdout:leaving image image_client.0-clone38 intact 2024-04-24T18:08:39.944 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:39.952+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:40.024 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:40.025 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:40.025 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:40.035 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:40.044 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #37, image image_client.0-clone37 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent38 2024-04-24T18:08:40.044 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:40.053+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:40.045 INFO:teuthology.orchestra.run.smithi049.stdout:3933 read 0x80ce5c2 thru 0x80d2ff1 (0x4a30 bytes) 2024-04-24T18:08:40.056 INFO:teuthology.orchestra.run.smithi049.stdout:3934 writesame 0xe4d8ef5 thru 0xe4d92b2 data_size 0x1(0x3be bytes) 2024-04-24T18:08:40.056 INFO:teuthology.orchestra.run.smithi049.stdout:3935 write 0x6f2ccfd thru 0x6f32176 (0x547a bytes) 2024-04-24T18:08:40.060 INFO:teuthology.orchestra.run.smithi049.stdout:3936 punch from 0xdc067b9 to 0xdc06f85, (0x7cc bytes) 2024-04-24T18:08:40.060 INFO:teuthology.orchestra.run.smithi049.stdout:3937 write 0xa8195a6 thru 0xa827a02 (0xe45d bytes) 2024-04-24T18:08:40.064 INFO:teuthology.orchestra.run.smithi049.stdout:3938 write 0xc798313 thru 0xc79ff9b (0x7c89 bytes) 2024-04-24T18:08:40.067 INFO:teuthology.orchestra.run.smithi049.stdout:3939 read 0x7ca5601 thru 0x7caaa73 (0x5473 bytes) 2024-04-24T18:08:40.067 INFO:teuthology.orchestra.run.smithi049.stdout:3940 read 0xb29ac7b thru 0xb2a6025 (0xb3ab bytes) 2024-04-24T18:08:40.067 INFO:teuthology.orchestra.run.smithi049.stdout:3941 punch from 0x1bbd687 to 0x1bc3c3f, (0x65b8 bytes) 2024-04-24T18:08:40.067 INFO:teuthology.orchestra.run.smithi049.stdout:3942 punch from 0xf10304 to 0xf10881, (0x57d bytes) 2024-04-24T18:08:40.067 INFO:teuthology.orchestra.run.smithi049.stdout:3945 writesame 0x38bf6da thru 0x38c0846 data_size 0x1(0x116d bytes) 2024-04-24T18:08:40.071 INFO:teuthology.orchestra.run.smithi049.stdout:3948 clone 40 order 23 su 65536 sc 14 2024-04-24T18:08:40.146 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:40.154+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:40.166 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:40.167 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:40.167 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:40.177 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:40.261 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:40.269+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.649571324999997, 2024-04-24T18:08:40.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:08:40.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:08:40.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:40.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 43.737096518999998, 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:08:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:08:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:08:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:08:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:08:40.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:08:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:08:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:08:40.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:08:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:08:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:08:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:08:40.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:08:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:08:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:08:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:08:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:08:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:08:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3096279959999997, 2024-04-24T18:08:40.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:08:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:08:40.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:08:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:08:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:08:40.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:08:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:08:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:08:40.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:08:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:08:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3096067109999998, 2024-04-24T18:08:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:08:40.372 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:08:40.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3095893480000003, 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.377 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:08:40.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3095439549999996, 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:08:40.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:08:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:08:40.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:08:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.386 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3095164539999997, 2024-04-24T18:08:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:08:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:08:40.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:40.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:40.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:08:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:08:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:08:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:08:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:08:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:08:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:08:40.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3094978939999997, 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.396 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:08:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:40.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.401 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3094740060000003, 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:08:40.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:08:40.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:08:40.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3094561120000003, 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.409 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3094214700000002, 2024-04-24T18:08:40.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:08:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.411 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:08:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:40.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:40.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:08:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:08:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:08:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:08:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:08:40.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:08:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:08:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.418 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3093269420000002, 2024-04-24T18:08:40.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.420 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:08:40.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:08:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:08:40.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:08:40.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:08:40.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.309308261, 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:40.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:08:40.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3092527819999997, 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.430 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.433 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3092183960000003, 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.434 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:08:40.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:08:40.437 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3091964369999998, 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.438 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3091793540000003, 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.442 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:08:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:08:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.309162894, 2024-04-24T18:08:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:08:40.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:08:40.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:08:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:08:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:08:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:08:40.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:08:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:08:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:08:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:08:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:08:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:08:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3090889790000002, 2024-04-24T18:08:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:08:40.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:08:40.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:40.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:08:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:08:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:08:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:08:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:08:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:08:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:08:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:08:40.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:08:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:08:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:08:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.464 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:40.465 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.3090364560000003, 2024-04-24T18:08:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:08:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:40.466 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:08:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:40.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:08:40.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:08:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:08:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:08:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:08:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:08:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:40.472 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:08:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:08:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:08:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:08:40.473 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:40.474 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:40.474 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:40.475 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:40.355+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:40.476 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:40.517 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:40.525+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:40.579 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:40.587+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:40.646 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:40.647 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:40.647 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:40.657 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:40.700 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:40.708+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:40.780 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:40.788+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:40.785 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:40.785 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:40.785 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:40.785 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:40.785 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:40.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:40.881 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:40.888+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:40.903 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone39 from 0xe4d92b3 (overlap 0xdad1a48) to 0x0 2024-04-24T18:08:40.981 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:40.989+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:40.999 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:40.999 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:40.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:40.999 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534952691999999, 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.000 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:08:41.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534934487999999, 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:08:41.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:08:41.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.009 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534913073, 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.014 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534895922, 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:08:41.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534879745, 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:08:41.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534861908, 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.024 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534831305000001, 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:08:41.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:08:41.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.033 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534811145999999, 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:41.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534794869000001, 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.039 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:08:41.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534777278, 2024-04-24T18:08:41.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:08:41.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.048 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534761409, 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:08:41.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:08:41.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534742863, 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:08:41.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:08:41.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:41.058 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534726818999999, 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:41.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:08:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534709527, 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:08:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534687870999999, 2024-04-24T18:08:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:08:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534669041000001, 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534652705999999, 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:08:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534636881000001, 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534618457000001, 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:08:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 14.534602081999999, 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:41.093 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:41.094 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:41.095 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #38, image image_client.0-clone38 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent39 2024-04-24T18:08:41.095 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:41.102+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:41.195 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:41.204+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:41.221 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:41.269 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:41.269 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:41.269 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:41.269 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:41.280 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:41.293 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:41.296 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:41.304+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:41.309 INFO:teuthology.orchestra.run.smithi049.stdout:3950 trunc from 0xe4d92b3 to 0x9f1708 2024-04-24T18:08:41.397 INFO:teuthology.orchestra.run.smithi049.stdout:3951 read 0x75751a thru 0x75d07f (0x5b66 bytes) 2024-04-24T18:08:41.397 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:41.405+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:41.410 INFO:teuthology.orchestra.run.smithi049.stdout:3952 writesame 0x4e6f848 thru 0x4e73917 data_size 0x22(0x40d0 bytes) 2024-04-24T18:08:41.415 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:41.416 INFO:teuthology.orchestra.run.smithi049.stdout:3953 trunc from 0x4e73918 to 0x12f84a0 2024-04-24T18:08:41.434 INFO:teuthology.orchestra.run.smithi049.stdout:3954 writesame 0x35ba81d thru 0x35c8688 data_size 0x3c(0xde6c bytes) 2024-04-24T18:08:41.438 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:41.450 INFO:teuthology.orchestra.run.smithi049.stdout:3956 write 0xbda578c thru 0xbda77ca (0x203f bytes) 2024-04-24T18:08:41.454 INFO:teuthology.orchestra.run.smithi049.stdout:3958 writesame 0x4e58183 thru 0x4e5f52e data_size 0xb(0x73ac bytes) 2024-04-24T18:08:41.457 INFO:teuthology.orchestra.run.smithi049.stdout:3959 writesame 0xa5e9473 thru 0xa5efa49 data_size 0x1d(0x65d7 bytes) 2024-04-24T18:08:41.458 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:41.460 INFO:teuthology.orchestra.run.smithi049.stdout:3960 write 0xc8e4d7 thru 0xc8ef54 (0xa7e bytes) 2024-04-24T18:08:41.460 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:41.461 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:41.461 INFO:teuthology.orchestra.run.smithi049.stdout:3961 write 0x5ef720 thru 0x5f6ed2 (0x77b3 bytes) 2024-04-24T18:08:41.463 INFO:teuthology.orchestra.run.smithi049.stdout:3962 punch from 0xba9f2b4 to 0xbaa3085, (0x3dd1 bytes) 2024-04-24T18:08:41.475 INFO:teuthology.orchestra.run.smithi049.stdout:3963 trunc from 0xbda77cb to 0x148bf65 2024-04-24T18:08:41.480 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:41.480 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:41.490 INFO:teuthology.orchestra.run.smithi049.stdout:3964 read 0x805b81 thru 0x806a77 (0xef7 bytes) 2024-04-24T18:08:41.491 INFO:teuthology.orchestra.run.smithi049.stdout:3965 read 0xb1bc8b thru 0xb1c52d (0x8a3 bytes) 2024-04-24T18:08:41.491 INFO:teuthology.orchestra.run.smithi049.stdout:3967 punch from 0x1a14c0 to 0x1ada56, (0xc596 bytes) 2024-04-24T18:08:41.498 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:41.506+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:41.500 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-04-24T18:08:41.500 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:08:41.500 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:08:41.501 INFO:teuthology.orchestra.run.smithi049.stdout:3968 write 0x46db3bb thru 0x46e685c (0xb4a2 bytes) 2024-04-24T18:08:41.503 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:41.503 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:41.515 INFO:teuthology.orchestra.run.smithi049.stdout:3970 writesame 0x9d593f3 thru 0x9d68b0a data_size 0x4(0xf718 bytes) 2024-04-24T18:08:41.526 INFO:teuthology.orchestra.run.smithi049.stdout:3971 write 0xcb67061 thru 0xcb6ca52 (0x59f2 bytes) 2024-04-24T18:08:41.526 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:41.530 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:41.530 INFO:teuthology.orchestra.run.smithi049.stdout:3974 writesame 0xb639a55 thru 0xb63f05e data_size 0x3(0x560a bytes) 2024-04-24T18:08:41.536 INFO:teuthology.orchestra.run.smithi049.stdout:3975 read 0x298d447 thru 0x2995d25 (0x88df bytes) 2024-04-24T18:08:41.537 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:41.537 INFO:teuthology.orchestra.run.smithi049.stdout:3977 read 0xa073114 thru 0xa077171 (0x405e bytes) 2024-04-24T18:08:41.537 INFO:teuthology.orchestra.run.smithi049.stdout:3978 read 0x4ae59a5 thru 0x4af3734 (0xdd90 bytes) 2024-04-24T18:08:41.538 INFO:teuthology.orchestra.run.smithi049.stdout:3980 write 0x75ac394 thru 0x75ba2ad (0xdf1a bytes) 2024-04-24T18:08:41.549 INFO:teuthology.orchestra.run.smithi049.stdout:3982 trunc from 0xcb6ca53 to 0x63916f0 2024-04-24T18:08:41.550 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:41.550 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:41.550 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:41.550 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:41.551 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:41.554 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:41.555 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:41.564 INFO:teuthology.orchestra.run.smithi049.stdout:3983 punch from 0x41b9be to 0x4245fc, (0x8c3e bytes) 2024-04-24T18:08:41.564 INFO:teuthology.orchestra.run.smithi049.stdout:3984 writesame 0x201125 thru 0x20517c data_size 0x47(0x4058 bytes) 2024-04-24T18:08:41.565 INFO:teuthology.orchestra.run.smithi049.stdout:3985 clone 41 order 21 su 1048576 sc 13 2024-04-24T18:08:41.599 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:41.607+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:41.642 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:41.643 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:41.656 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:41.740 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:41.748+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:41.800 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:41.809+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:41.829 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:41.829 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:41.841 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:41.845 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:08:41.860 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:08:41.862 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:08:41.877 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":121,"stamp":"2024-04-24T18:08:41.838191+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":961542799,"num_objects":2373,"num_object_clones":561,"num_object_copies":4746,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2371,"num_whiteouts":1310,"num_read":173807,"num_read_kb":598780,"num_write":14874,"num_write_kb":349750,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":305,"num_bytes_recovered":18486782,"num_keys_recovered":80,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12593,"ondisk_log_size":12593,"up":80,"acting":80,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2504844,"kb_used_data":60412,"kb_used_omap":13,"kb_used_meta":2444274,"kb_avail":752469876,"statfs":{"total":773094113280,"available":770529153024,"internally_reserved":0,"allocated":61861888,"data_stored":537841724,"data_compressed":10832579,"data_compressed_allocated":34390016,"data_compressed_original":517450398,"omap_allocated":13632,"internal_metadata":2502937280},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":51894500,"num_objects":187,"num_object_clones":42,"num_object_copies":374,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":33,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":187,"num_whiteouts":51,"num_read":14709,"num_read_kb":43377,"num_write":1421,"num_write_kb":21120,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.525357"},"pg_stats":[{"pgid":"4.1f","version":"81'315","reported_seq":4557,"reported_epoch":81,"state":"active+undersized+degraded","last_fresh":"2024-04-24T18:08:36.433035+0000","last_change":"2024-04-24T18:08:22.745997+0000","last_active":"2024-04-24T18:08:36.433035+0000","last_peered":"2024-04-24T18:08:36.433035+0000","last_clean":"2024-04-24T18:08:18.970509+0000","last_became_active":"2024-04-24T18:08:22.745997+0000","last_became_peered":"2024-04-24T18:08:22.745997+0000","last_unstale":"2024-04-24T18:08:36.433035+0000","last_undegraded":"2024-04-24T18:08:22.745581+0000","last_fullsized":"2024-04-24T18:08:22.745501+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":315,"log_dups_size":0,"ondisk_log_size":315,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:33:55.601843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":39059628,"num_objects":89,"num_object_clones":20,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":89,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":54,"num_read":4937,"num_read_kb":22750,"num_write":335,"num_write_kb":8262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5],"acting":[5],"avail_no_missing":["5"],"object_location_counts":[{"shards":"5","objects":89}],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"81'1010","reported_seq":4536,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.612469+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:08:36.612469+0000","last_peered":"2024-04-24T18:08:36.612469+0000","last_clean":"2024-04-24T18:08:36.612469+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:08:36.612469+0000","last_undegraded":"2024-04-24T18:08:36.612469+0000","last_fullsized":"2024-04-24T18:08:36.612469+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1010,"log_dups_size":0,"ondisk_log_size":1010,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":28883756,"num_objects":102,"num_object_clones":21,"num_object_copies":204,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":60,"num_read":7966,"num_read_kb":16360,"num_write":1585,"num_write_kb":8060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"81'256","reported_seq":2877,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.586845+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:08:36.586845+0000","last_peered":"2024-04-24T18:08:36.586845+0000","last_clean":"2024-04-24T18:08:36.586845+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:08:36.586845+0000","last_undegraded":"2024-04-24T18:08:36.586845+0000","last_fullsized":"2024-04-24T18:08:36.586845+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":256,"log_dups_size":0,"ondisk_log_size":256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":16445114,"num_objects":82,"num_object_clones":19,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":47,"num_read":2891,"num_read_kb":13396,"num_write":233,"num_write_kb":9899,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"81'659","reported_seq":6793,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.609270+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:08:36.609270+0000","last_peered":"2024-04-24T18:08:36.609270+0000","last_clean":"2024-04-24T18:08:36.609270+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:08:36.609270+0000","last_undegraded":"2024-04-24T18:08:36.609270+0000","last_fullsized":"2024-04-24T18:08:36.609270+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":659,"log_dups_size":0,"ondisk_log_size":659,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":26508425,"num_objects":66,"num_object_clones":18,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":32,"num_read":8726,"num_read_kb":18386,"num_write":971,"num_write_kb":7300,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"81'265","reported_seq":5008,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.658487+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:08:36.658487+0000","last_peered":"2024-04-24T18:08:36.658487+0000","last_clean":"2024-04-24T18:08:36.658487+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:08:36.658487+0000","last_undegraded":"2024-04-24T18:08:36.658487+0000","last_fullsized":"2024-04-24T18:08:36.658487+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":265,"log_dups_size":0,"ondisk_log_size":265,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":42295852,"num_objects":78,"num_object_clones":22,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":33,"num_read":4809,"num_read_kb":26589,"num_write":271,"num_write_kb":14431,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"81'290","reported_seq":8479,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.481881+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:08:36.481881+0000","last_peered":"2024-04-24T18:08:36.481881+0000","last_clean":"2024-04-24T18:08:36.481881+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:08:36.481881+0000","last_undegraded":"2024-04-24T18:08:36.481881+0000","last_fullsized":"2024-04-24T18:08:36.481881+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":25914708,"num_objects":74,"num_object_clones":21,"num_object_copies":148,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":74,"num_whiteouts":37,"num_read":8954,"num_read_kb":21820,"num_write":250,"num_write_kb":10532,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"81'433","reported_seq":7122,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.635260+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:08:36.635260+0000","last_peered":"2024-04-24T18:08:36.635260+0000","last_clean":"2024-04-24T18:08:36.635260+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:08:36.635260+0000","last_undegraded":"2024-04-24T18:08:36.635260+0000","last_fullsized":"2024-04-24T18:08:36.635260+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":433,"log_dups_size":0,"ondisk_log_size":433,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":24960114,"num_objects":82,"num_object_clones":16,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":48,"num_read":7999,"num_read_kb":23399,"num_write":478,"num_write_kb":6967,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"80'271","reported_seq":6585,"reported_epoch":81,"state":"active+undersized+degraded","last_fresh":"2024-04-24T18:08:36.465711+0000","last_change":"2024-04-24T18:08:22.743834+0000","last_active":"2024-04-24T18:08:36.465711+0000","last_peered":"2024-04-24T18:08:36.465711+0000","last_clean":"2024-04-24T18:08:20.733439+0000","last_became_active":"2024-04-24T18:08:22.743834+0000","last_became_peered":"2024-04-24T18:08:22.743834+0000","last_unstale":"2024-04-24T18:08:36.465711+0000","last_undegraded":"2024-04-24T18:08:22.743158+0000","last_fullsized":"2024-04-24T18:08:22.743069+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":271,"log_dups_size":0,"ondisk_log_size":271,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:41:37.885328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":33630464,"num_objects":64,"num_object_clones":15,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":64,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":35,"num_read":7212,"num_read_kb":22867,"num_write":267,"num_write_kb":24111,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":64}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"81'287","reported_seq":3696,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.446651+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:08:36.446651+0000","last_peered":"2024-04-24T18:08:36.446651+0000","last_clean":"2024-04-24T18:08:36.446651+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:08:36.446651+0000","last_undegraded":"2024-04-24T18:08:36.446651+0000","last_fullsized":"2024-04-24T18:08:36.446651+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":287,"log_dups_size":0,"ondisk_log_size":287,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":30071986,"num_objects":80,"num_object_clones":23,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":48,"num_read":3475,"num_read_kb":15525,"num_write":286,"num_write_kb":13808,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"81'848","reported_seq":2157,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.440904+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:08:36.440904+0000","last_peered":"2024-04-24T18:08:36.440904+0000","last_clean":"2024-04-24T18:08:36.440904+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:08:36.440904+0000","last_undegraded":"2024-04-24T18:08:36.440904+0000","last_fullsized":"2024-04-24T18:08:36.440904+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":848,"log_dups_size":0,"ondisk_log_size":848,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":28870405,"num_objects":84,"num_object_clones":17,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":45,"num_read":5827,"num_read_kb":17579,"num_write":1320,"num_write_kb":9964,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":20,"reported_epoch":80,"state":"active+undersized","last_fresh":"2024-04-24T18:08:34.812870+0000","last_change":"2024-04-24T18:08:22.745079+0000","last_active":"2024-04-24T18:08:34.812870+0000","last_peered":"2024-04-24T18:08:34.812870+0000","last_clean":"2024-04-24T18:07:19.379531+0000","last_became_active":"2024-04-24T18:08:22.745079+0000","last_became_peered":"2024-04-24T18:08:22.745079+0000","last_unstale":"2024-04-24T18:08:34.812870+0000","last_undegraded":"2024-04-24T18:08:34.812870+0000","last_fullsized":"2024-04-24T18:08:22.744744+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:27:39.930984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3],"acting":[3],"avail_no_missing":["3"],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":21,"reported_epoch":81,"state":"active+undersized","last_fresh":"2024-04-24T18:08:35.808522+0000","last_change":"2024-04-24T18:08:22.740627+0000","last_active":"2024-04-24T18:08:35.808522+0000","last_peered":"2024-04-24T18:08:35.808522+0000","last_clean":"2024-04-24T18:07:19.379531+0000","last_became_active":"2024-04-24T18:08:22.740627+0000","last_became_peered":"2024-04-24T18:08:22.740627+0000","last_unstale":"2024-04-24T18:08:35.808522+0000","last_undegraded":"2024-04-24T18:08:35.808522+0000","last_fullsized":"2024-04-24T18:08:22.739972+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:55.050690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0],"acting":[0],"avail_no_missing":["0"],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"81'374","reported_seq":3371,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.447247+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:08:36.447247+0000","last_peered":"2024-04-24T18:08:36.447247+0000","last_clean":"2024-04-24T18:08:36.447247+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:08:36.447247+0000","last_undegraded":"2024-04-24T18:08:36.447247+0000","last_fullsized":"2024-04-24T18:08:36.447247+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":374,"log_dups_size":0,"ondisk_log_size":374,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":54773735,"num_objects":82,"num_object_clones":23,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":43,"num_read":3688,"num_read_kb":25408,"num_write":449,"num_write_kb":17103,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":71,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.807512+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:35.807512+0000","last_peered":"2024-04-24T18:08:35.807512+0000","last_clean":"2024-04-24T18:08:35.807512+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:35.807512+0000","last_undegraded":"2024-04-24T18:08:35.807512+0000","last_fullsized":"2024-04-24T18:08:35.807512+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.845418+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:35.845418+0000","last_peered":"2024-04-24T18:08:35.845418+0000","last_clean":"2024-04-24T18:08:35.845418+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:35.845418+0000","last_undegraded":"2024-04-24T18:08:35.845418+0000","last_fullsized":"2024-04-24T18:08:35.845418+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"81'412","reported_seq":3823,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.917089+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:08:35.917089+0000","last_peered":"2024-04-24T18:08:35.917089+0000","last_clean":"2024-04-24T18:08:35.917089+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:08:35.917089+0000","last_undegraded":"2024-04-24T18:08:35.917089+0000","last_fullsized":"2024-04-24T18:08:35.917089+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":412,"log_dups_size":0,"ondisk_log_size":412,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":14067970,"num_objects":70,"num_object_clones":15,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":47,"num_read":5181,"num_read_kb":10208,"num_write":431,"num_write_kb":5002,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"61'1","reported_seq":71,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T18:08:34.812764+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:34.812764+0000","last_peered":"2024-04-24T18:08:34.812764+0000","last_clean":"2024-04-24T18:08:34.812764+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:34.812764+0000","last_undegraded":"2024-04-24T18:08:34.812764+0000","last_fullsized":"2024-04-24T18:08:34.812764+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:06.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":72,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.296138+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:35.296138+0000","last_peered":"2024-04-24T18:08:35.296138+0000","last_clean":"2024-04-24T18:08:35.296138+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:35.296138+0000","last_undegraded":"2024-04-24T18:08:35.296138+0000","last_fullsized":"2024-04-24T18:08:35.296138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"81'570","reported_seq":5607,"reported_epoch":81,"state":"active+undersized+degraded","last_fresh":"2024-04-24T18:08:36.466304+0000","last_change":"2024-04-24T18:08:22.744620+0000","last_active":"2024-04-24T18:08:36.466304+0000","last_peered":"2024-04-24T18:08:36.466304+0000","last_clean":"2024-04-24T18:08:19.722254+0000","last_became_active":"2024-04-24T18:08:22.744620+0000","last_became_peered":"2024-04-24T18:08:22.744620+0000","last_unstale":"2024-04-24T18:08:36.466304+0000","last_undegraded":"2024-04-24T18:08:22.743836+0000","last_fullsized":"2024-04-24T18:08:22.743758+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":570,"log_dups_size":0,"ondisk_log_size":570,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:28:13.504832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":35375215,"num_objects":78,"num_object_clones":16,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":78,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":39,"num_read":7946,"num_read_kb":35593,"num_write":702,"num_write_kb":23447,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":78}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":45,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.807059+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:35.807059+0000","last_peered":"2024-04-24T18:08:35.807059+0000","last_clean":"2024-04-24T18:08:35.807059+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:35.807059+0000","last_undegraded":"2024-04-24T18:08:35.807059+0000","last_fullsized":"2024-04-24T18:08:35.807059+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":72,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.296404+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:35.296404+0000","last_peered":"2024-04-24T18:08:35.296404+0000","last_clean":"2024-04-24T18:08:35.296404+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:35.296404+0000","last_undegraded":"2024-04-24T18:08:35.296404+0000","last_fullsized":"2024-04-24T18:08:35.296404+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"81'257","reported_seq":3283,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.894889+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:08:35.894889+0000","last_peered":"2024-04-24T18:08:35.894889+0000","last_clean":"2024-04-24T18:08:35.894889+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:08:35.894889+0000","last_undegraded":"2024-04-24T18:08:35.894889+0000","last_fullsized":"2024-04-24T18:08:35.894889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":257,"log_dups_size":0,"ondisk_log_size":257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":25590980,"num_objects":81,"num_object_clones":23,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":44,"num_read":4605,"num_read_kb":22377,"num_write":377,"num_write_kb":11959,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'1","reported_seq":72,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.808426+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:35.808426+0000","last_peered":"2024-04-24T18:08:35.808426+0000","last_clean":"2024-04-24T18:08:35.808426+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:35.808426+0000","last_undegraded":"2024-04-24T18:08:35.808426+0000","last_fullsized":"2024-04-24T18:08:35.808426+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":270,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.296156+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:35.296156+0000","last_peered":"2024-04-24T18:08:35.296156+0000","last_clean":"2024-04-24T18:08:35.296156+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:35.296156+0000","last_undegraded":"2024-04-24T18:08:35.296156+0000","last_fullsized":"2024-04-24T18:08:35.296156+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":51,"reported_epoch":81,"state":"active+undersized","last_fresh":"2024-04-24T18:08:35.807277+0000","last_change":"2024-04-24T18:08:22.740918+0000","last_active":"2024-04-24T18:08:35.807277+0000","last_peered":"2024-04-24T18:08:35.807277+0000","last_clean":"2024-04-24T18:08:20.730259+0000","last_became_active":"2024-04-24T18:08:22.740918+0000","last_became_peered":"2024-04-24T18:08:22.740918+0000","last_unstale":"2024-04-24T18:08:35.807277+0000","last_undegraded":"2024-04-24T18:08:35.807277+0000","last_fullsized":"2024-04-24T18:08:22.740152+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:50:11.742589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"81'342","reported_seq":6521,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.453673+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:08:36.453673+0000","last_peered":"2024-04-24T18:08:36.453673+0000","last_clean":"2024-04-24T18:08:36.453673+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:08:36.453673+0000","last_undegraded":"2024-04-24T18:08:36.453673+0000","last_fullsized":"2024-04-24T18:08:36.453673+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":342,"log_dups_size":0,"ondisk_log_size":342,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":25507892,"num_objects":63,"num_object_clones":13,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":34,"num_read":8205,"num_read_kb":18762,"num_write":340,"num_write_kb":12250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"81'254","reported_seq":3524,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.584200+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:08:36.584200+0000","last_peered":"2024-04-24T18:08:36.584200+0000","last_clean":"2024-04-24T18:08:36.584200+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:08:36.584200+0000","last_undegraded":"2024-04-24T18:08:36.584200+0000","last_fullsized":"2024-04-24T18:08:36.584200+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":254,"log_dups_size":0,"ondisk_log_size":254,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":32571549,"num_objects":59,"num_object_clones":12,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":38,"num_read":3899,"num_read_kb":20386,"num_write":239,"num_write_kb":19060,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":73,"reported_epoch":80,"state":"active+clean","last_fresh":"2024-04-24T18:08:34.812625+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:34.812625+0000","last_peered":"2024-04-24T18:08:34.812625+0000","last_clean":"2024-04-24T18:08:34.812625+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:34.812625+0000","last_undegraded":"2024-04-24T18:08:34.812625+0000","last_fullsized":"2024-04-24T18:08:34.812625+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"81'238","reported_seq":2746,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.530889+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:08:36.530889+0000","last_peered":"2024-04-24T18:08:36.530889+0000","last_clean":"2024-04-24T18:08:36.530889+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:08:36.530889+0000","last_undegraded":"2024-04-24T18:08:36.530889+0000","last_fullsized":"2024-04-24T18:08:36.530889+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":35323827,"num_objects":71,"num_object_clones":18,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":36,"num_read":2632,"num_read_kb":10998,"num_write":218,"num_write_kb":6056,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":74,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.807562+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:35.807562+0000","last_peered":"2024-04-24T18:08:35.807562+0000","last_clean":"2024-04-24T18:08:35.807562+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:35.807562+0000","last_undegraded":"2024-04-24T18:08:35.807562+0000","last_fullsized":"2024-04-24T18:08:35.807562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"81'340","reported_seq":6756,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.440056+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:08:36.440056+0000","last_peered":"2024-04-24T18:08:36.440056+0000","last_clean":"2024-04-24T18:08:36.440056+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:08:36.440056+0000","last_undegraded":"2024-04-24T18:08:36.440056+0000","last_fullsized":"2024-04-24T18:08:36.440056+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":340,"log_dups_size":0,"ondisk_log_size":340,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":33430815,"num_objects":78,"num_object_clones":20,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":40,"num_read":7532,"num_read_kb":31143,"num_write":339,"num_write_kb":10891,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"82'284","reported_seq":3991,"reported_epoch":82,"state":"active+clean","last_fresh":"2024-04-24T18:08:37.243040+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:08:37.243040+0000","last_peered":"2024-04-24T18:08:37.243040+0000","last_clean":"2024-04-24T18:08:37.243040+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:08:37.243040+0000","last_undegraded":"2024-04-24T18:08:37.243040+0000","last_fullsized":"2024-04-24T18:08:37.243040+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":34809668,"num_objects":72,"num_object_clones":15,"num_object_copies":144,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":72,"num_whiteouts":34,"num_read":4155,"num_read_kb":13272,"num_write":340,"num_write_kb":5807,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"81'407","reported_seq":1062,"reported_epoch":81,"state":"active+undersized+degraded","last_fresh":"2024-04-24T18:08:36.370726+0000","last_change":"2024-04-24T18:08:22.740983+0000","last_active":"2024-04-24T18:08:36.370726+0000","last_peered":"2024-04-24T18:08:36.370726+0000","last_clean":"2024-04-24T18:08:18.932949+0000","last_became_active":"2024-04-24T18:08:22.740983+0000","last_became_peered":"2024-04-24T18:08:22.740983+0000","last_unstale":"2024-04-24T18:08:36.370726+0000","last_undegraded":"2024-04-24T18:08:22.739902+0000","last_fullsized":"2024-04-24T18:08:22.739805+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":407,"log_dups_size":0,"ondisk_log_size":407,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:01:19.972636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":14814482,"num_objects":47,"num_object_clones":5,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":47,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":28,"num_read":1584,"num_read_kb":3968,"num_write":524,"num_write_kb":4099,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":1623115,"num_keys_recovered":4,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":47}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.b","version":"81'380","reported_seq":3147,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.447367+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:08:36.447367+0000","last_peered":"2024-04-24T18:08:36.447367+0000","last_clean":"2024-04-24T18:08:36.447367+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:08:36.447367+0000","last_undegraded":"2024-04-24T18:08:36.447367+0000","last_fullsized":"2024-04-24T18:08:36.447367+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":380,"log_dups_size":0,"ondisk_log_size":380,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":22447192,"num_objects":62,"num_object_clones":14,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":35,"num_read":5463,"num_read_kb":11569,"num_write":428,"num_write_kb":7480,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"81'282","reported_seq":4033,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.481891+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:08:36.481891+0000","last_peered":"2024-04-24T18:08:36.481891+0000","last_clean":"2024-04-24T18:08:36.481891+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:08:36.481891+0000","last_undegraded":"2024-04-24T18:08:36.481891+0000","last_fullsized":"2024-04-24T18:08:36.481891+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":24836007,"num_objects":55,"num_object_clones":12,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":30,"num_read":4902,"num_read_kb":17819,"num_write":303,"num_write_kb":22543,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"81'284","reported_seq":4460,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.599070+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:08:36.599070+0000","last_peered":"2024-04-24T18:08:36.599070+0000","last_clean":"2024-04-24T18:08:36.599070+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:08:36.599070+0000","last_undegraded":"2024-04-24T18:08:36.599070+0000","last_fullsized":"2024-04-24T18:08:36.599070+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":284,"log_dups_size":0,"ondisk_log_size":284,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":32881591,"num_objects":87,"num_object_clones":21,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":50,"num_read":4489,"num_read_kb":11278,"num_write":226,"num_write_kb":4076,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"81'302","reported_seq":2744,"reported_epoch":81,"state":"active+undersized+degraded","last_fresh":"2024-04-24T18:08:36.636287+0000","last_change":"2024-04-24T18:08:22.744241+0000","last_active":"2024-04-24T18:08:36.636287+0000","last_peered":"2024-04-24T18:08:36.636287+0000","last_clean":"2024-04-24T18:08:20.733646+0000","last_became_active":"2024-04-24T18:08:22.744241+0000","last_became_peered":"2024-04-24T18:08:22.744241+0000","last_unstale":"2024-04-24T18:08:36.636287+0000","last_undegraded":"2024-04-24T18:08:22.743417+0000","last_fullsized":"2024-04-24T18:08:22.743344+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":302,"log_dups_size":0,"ondisk_log_size":302,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:38:27.821013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":26645934,"num_objects":69,"num_object_clones":18,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":69,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":37,"num_read":3167,"num_read_kb":8615,"num_write":298,"num_write_kb":7552,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":69}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"81'493","reported_seq":2789,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.441856+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:08:36.441856+0000","last_peered":"2024-04-24T18:08:36.441856+0000","last_clean":"2024-04-24T18:08:36.441856+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:08:36.441856+0000","last_undegraded":"2024-04-24T18:08:36.441856+0000","last_fullsized":"2024-04-24T18:08:36.441856+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":20700276,"num_objects":70,"num_object_clones":13,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":37,"num_read":4591,"num_read_kb":16130,"num_write":616,"num_write_kb":6205,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"81'306","reported_seq":4546,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.446115+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:08:36.446115+0000","last_peered":"2024-04-24T18:08:36.446115+0000","last_clean":"2024-04-24T18:08:36.446115+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:08:36.446115+0000","last_undegraded":"2024-04-24T18:08:36.446115+0000","last_fullsized":"2024-04-24T18:08:36.446115+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":306,"log_dups_size":0,"ondisk_log_size":306,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":20025199,"num_objects":78,"num_object_clones":22,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":43,"num_read":4936,"num_read_kb":12504,"num_write":278,"num_write_kb":7695,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"81'429","reported_seq":6408,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.879640+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:08:35.879640+0000","last_peered":"2024-04-24T18:08:35.879640+0000","last_clean":"2024-04-24T18:08:35.879640+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:08:35.879640+0000","last_undegraded":"2024-04-24T18:08:35.879640+0000","last_fullsized":"2024-04-24T18:08:35.879640+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":429,"log_dups_size":0,"ondisk_log_size":429,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":81024381,"num_objects":88,"num_object_clones":23,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":51,"num_read":7190,"num_read_kb":40753,"num_write":526,"num_write_kb":22041,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"81'328","reported_seq":4095,"reported_epoch":81,"state":"active+undersized+degraded","last_fresh":"2024-04-24T18:08:36.617347+0000","last_change":"2024-04-24T18:08:22.741007+0000","last_active":"2024-04-24T18:08:36.617347+0000","last_peered":"2024-04-24T18:08:36.617347+0000","last_clean":"2024-04-24T18:08:20.798811+0000","last_became_active":"2024-04-24T18:08:22.741007+0000","last_became_peered":"2024-04-24T18:08:22.741007+0000","last_unstale":"2024-04-24T18:08:36.617347+0000","last_undegraded":"2024-04-24T18:08:22.740312+0000","last_fullsized":"2024-04-24T18:08:22.740229+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":328,"log_dups_size":0,"ondisk_log_size":328,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:34:12.379010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":24577615,"num_objects":81,"num_object_clones":15,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":81,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":47,"num_read":7377,"num_read_kb":27847,"num_write":285,"num_write_kb":7746,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":27,"num_bytes_recovered":2974171,"num_keys_recovered":9,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[{"shards":"1","objects":81}],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"81'238","reported_seq":5044,"reported_epoch":81,"state":"active+undersized+degraded","last_fresh":"2024-04-24T18:08:36.634131+0000","last_change":"2024-04-24T18:08:22.744313+0000","last_active":"2024-04-24T18:08:36.634131+0000","last_peered":"2024-04-24T18:08:36.634131+0000","last_clean":"2024-04-24T18:08:20.733644+0000","last_became_active":"2024-04-24T18:08:22.744313+0000","last_became_peered":"2024-04-24T18:08:22.744313+0000","last_unstale":"2024-04-24T18:08:36.634131+0000","last_undegraded":"2024-04-24T18:08:22.743540+0000","last_fullsized":"2024-04-24T18:08:22.743445+0000","mapping_epoch":69,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":238,"log_dups_size":0,"ondisk_log_size":238,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:57:52.223264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":19405274,"num_objects":70,"num_object_clones":21,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":70,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":38,"num_read":4825,"num_read_kb":13277,"num_write":197,"num_write_kb":9296,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4],"acting":[4],"avail_no_missing":["4"],"object_location_counts":[{"shards":"4","objects":70}],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"81'326","reported_seq":5064,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:35.992340+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:08:35.992340+0000","last_peered":"2024-04-24T18:08:35.992340+0000","last_clean":"2024-04-24T18:08:35.992340+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:08:35.992340+0000","last_undegraded":"2024-04-24T18:08:35.992340+0000","last_fullsized":"2024-04-24T18:08:35.992340+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":326,"log_dups_size":0,"ondisk_log_size":326,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":36067704,"num_objects":59,"num_object_clones":15,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":31,"num_read":6617,"num_read_kb":18062,"num_write":299,"num_write_kb":5878,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"81'222","reported_seq":2828,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.439759+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:08:36.439759+0000","last_peered":"2024-04-24T18:08:36.439759+0000","last_clean":"2024-04-24T18:08:36.439759+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:08:36.439759+0000","last_undegraded":"2024-04-24T18:08:36.439759+0000","last_fullsized":"2024-04-24T18:08:36.439759+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":222,"log_dups_size":0,"ondisk_log_size":222,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":6921054,"num_objects":66,"num_object_clones":15,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":41,"num_read":3501,"num_read_kb":6398,"num_write":186,"num_write_kb":4169,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"81'772","reported_seq":2489,"reported_epoch":81,"state":"active+clean","last_fresh":"2024-04-24T18:08:36.455290+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:08:36.455290+0000","last_peered":"2024-04-24T18:08:36.455290+0000","last_clean":"2024-04-24T18:08:36.455290+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:08:36.455290+0000","last_undegraded":"2024-04-24T18:08:36.455290+0000","last_fullsized":"2024-04-24T18:08:36.455290+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":772,"log_dups_size":0,"ondisk_log_size":772,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":42513619,"num_objects":82,"num_object_clones":20,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":46,"num_read":4410,"num_read_kb":23642,"num_write":1118,"num_write_kb":13645,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":960952431,"num_objects":2369,"num_object_clones":561,"num_object_copies":4738,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":498,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2369,"num_whiteouts":1308,"num_read":173691,"num_read_kb":598680,"num_write":14715,"num_write_kb":347334,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":301,"num_bytes_recovered":17306046,"num_keys_recovered":80,"num_objects_omap":40,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":54308864,"data_stored":533009514,"data_compressed":10813970,"data_compressed_allocated":34312192,"data_compressed_original":516237982,"omap_allocated":0,"internal_metadata":0},"log_size":12474,"ondisk_log_size":12474,"up":57,"acting":57,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":13,"acting":13,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316188,"kb_used_data":6512,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94055652,"statfs":{"total":96636764160,"available":96312987648,"internally_reserved":0,"allocated":6668288,"data_stored":59842197,"data_compressed":945105,"data_compressed_allocated":3694592,"data_compressed_original":57833704,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,5],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35999999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":7.6740000000000004},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":7.3879999999999999},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":7.4749999999999996}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.33700000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.499},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.59899999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.54900000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309476,"kb_used_data":3924,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94062364,"statfs":{"total":96636764160,"available":96319860736,"internally_reserved":0,"allocated":4018176,"data_stored":29406206,"data_compressed":781662,"data_compressed_allocated":2088960,"data_compressed_original":28024501,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.251}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.52}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":6.8280000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":312640,"kb_used_data":11192,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94059200,"statfs":{"total":96636764160,"available":96316620800,"internally_reserved":0,"allocated":11460608,"data_stored":83003215,"data_compressed":1774201,"data_compressed_allocated":5709824,"data_compressed_original":78704524,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.15,"5min":14.15,"15min":14.15},"last":0.249}]},{"osd":2,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.28499999999999998},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":0.33500000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":14.089,"5min":14.089,"15min":14.089},"last":0.379},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.436}]},{"osd":5,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.42499999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":1.1990000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":14.051,"5min":14.051,"15min":14.051},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.44700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315004,"kb_used_data":5356,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94056836,"statfs":{"total":96636764160,"available":96314200064,"internally_reserved":0,"allocated":5484544,"data_stored":50907884,"data_compressed":916961,"data_compressed_allocated":3117056,"data_compressed_original":49313082,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.24399999999999999},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":26.936,"5min":26.936,"15min":26.936},"last":0.35399999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.33200000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":14.371,"5min":14.371,"15min":14.371},"last":0.58299999999999996},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.973000000000001,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.414,"5min":14.414,"15min":14.414},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.437}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":14.452,"5min":14.452,"15min":14.452},"last":0.40500000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.42499999999999999}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"last":0.371},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":14.486000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.45200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574866,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308856,"kb_used_data":3312,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94062984,"statfs":{"total":96636764160,"available":96320495616,"internally_reserved":0,"allocated":3391488,"data_stored":19511530,"data_compressed":452614,"data_compressed_allocated":1462272,"data_compressed_original":18222740,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.23799999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.27500000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.313}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.32600000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.37},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.38400000000000001}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.42199999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.52500000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.46000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309736,"kb_used_data":12368,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062104,"statfs":{"total":96636764160,"available":96319594496,"internally_reserved":0,"allocated":12664832,"data_stored":123173245,"data_compressed":2137338,"data_compressed_allocated":7512064,"data_compressed_original":119016401,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.308}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.32700000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.375}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.54200000000000004}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.46600000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574867,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":319368,"kb_used_data":9728,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94052472,"statfs":{"total":96636764160,"available":96309731328,"internally_reserved":0,"allocated":9961472,"data_stored":101289444,"data_compressed":2325730,"data_compressed_allocated":6209536,"data_compressed_original":98340115,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.439}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.39300000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.32700000000000001}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.27900000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.499},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.58399999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.315},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.371}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313576,"kb_used_data":8020,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94058264,"statfs":{"total":96636764160,"available":96315662336,"internally_reserved":0,"allocated":8212480,"data_stored":70708003,"data_compressed":1498968,"data_compressed_allocated":4595712,"data_compressed_original":67995331,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.54100000000000004}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39900000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.375},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.436}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.45400000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.31900000000000001},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.38500000000000001}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.48399999999999999}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.34200000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4517888,"data_stored":50438979,"data_compressed":916961,"data_compressed_allocated":3117056,"data_compressed_original":49313082,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10465280,"data_stored":81950124,"data_compressed":1764499,"data_compressed_allocated":5672960,"data_compressed_original":78114700,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2437120,"data_stored":19048807,"data_compressed":452614,"data_compressed_allocated":1462272,"data_compressed_original":18222740,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11710464,"data_stored":122710522,"data_compressed":2137338,"data_compressed_allocated":7512064,"data_compressed_original":119016401,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8994816,"data_stored":100820539,"data_compressed":2325730,"data_compressed_allocated":6209536,"data_compressed_original":98340115,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7245824,"data_stored":70239098,"data_compressed":1498968,"data_compressed_allocated":4595712,"data_compressed_original":67995331,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":3280896,"data_stored":29012339,"data_compressed":781419,"data_compressed_allocated":2084864,"data_compressed_original":27975349,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5656576,"data_stored":58789106,"data_compressed":936441,"data_compressed_allocated":3657728,"data_compressed_original":57260264,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:08:41.918 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:41.926+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:41.919 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone40 from 0x63916f0 (overlap 0x9f1708) to 0x8f2e54 2024-04-24T18:08:42.002 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:42.010+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:42.013 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:42.013 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:42.015 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #39, image image_client.0-clone39 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent40 2024-04-24T18:08:42.017 INFO:teuthology.orchestra.run.smithi049.stdout:3986 read 0x2618628 thru 0x261e4b7 (0x5e90 bytes) 2024-04-24T18:08:42.017 INFO:teuthology.orchestra.run.smithi049.stdout:3987 read 0x1443c84 thru 0x1453227 (0xf5a4 bytes) 2024-04-24T18:08:42.025 INFO:teuthology.orchestra.run.smithi049.stdout:3988 trunc from 0x63916f0 to 0xc44fbe2 2024-04-24T18:08:42.030 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:42.033 INFO:teuthology.orchestra.run.smithi049.stdout:3989 punch from 0x9429cfd to 0x942f6f1, (0x59f4 bytes) 2024-04-24T18:08:42.038 INFO:teuthology.orchestra.run.smithi049.stdout:3990 write 0xc457ee4 thru 0xc45f89b (0x79b8 bytes) 2024-04-24T18:08:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:3991 read 0xbaaabf0 thru 0xbab12c6 (0x66d7 bytes) 2024-04-24T18:08:42.040 INFO:teuthology.orchestra.run.smithi049.stdout:3992 write 0x166c80b thru 0x1670c2d (0x4423 bytes) 2024-04-24T18:08:42.046 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:42.047 INFO:teuthology.orchestra.run.smithi049.stdout:3995 clone 42 order 18 su 4096 sc 12 2024-04-24T18:08:42.103 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:42.111+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:42.163 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:42.163 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:42.173 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:42.216 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:42.225+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:42.298 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:42.298 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:42.304 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:42.313+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:42.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:42.406 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:42.414+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:42.470 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:42.471 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:42.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:42.506 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:42.515+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:42.604 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:42.604 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:42.607 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:42.616+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:42.614 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:42.708 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:42.717+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:42.779 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:42.779 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:42.788 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:42.831 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:42.839+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:42.838 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:08:42.910 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:42.918+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:42.923 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone41 from 0xc45f89c (overlap 0x63916f0) to 0x45b29a8 2024-04-24T18:08:42.971 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:42.971 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:42.981 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:43.024 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:43.033+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:43.100 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #40, image image_client.0-clone40 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent41 2024-04-24T18:08:43.108 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:43.109 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:43.111 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:43.119+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:43.118 INFO:teuthology.orchestra.run.smithi049.stdout:3996 trunc from 0xc45f89c to 0xe572332 2024-04-24T18:08:43.119 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:43.126 INFO:teuthology.orchestra.run.smithi049.stdout:3997 writesame 0x8887a88 thru 0x8894532 data_size 0xd(0xcaab bytes) 2024-04-24T18:08:43.139 INFO:teuthology.orchestra.run.smithi049.stdout:3998 punch from 0x357f8ab to 0x35882c6, (0x8a1b bytes) 2024-04-24T18:08:43.140 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.140 INFO:teuthology.orchestra.run.smithi049.stdout:4000 punch from 0xd24d09 to 0xd2eea5, (0xa19c bytes) 2024-04-24T18:08:43.140 INFO:teuthology.orchestra.run.smithi049.stdout:4001 write 0x4a60991 thru 0x4a6cc64 (0xc2d4 bytes) 2024-04-24T18:08:43.170 INFO:teuthology.orchestra.run.smithi049.stdout:4003 punch from 0xa2a8fa8 to 0xa2b1fad, (0x9005 bytes) 2024-04-24T18:08:43.170 INFO:teuthology.orchestra.run.smithi049.stdout:4004 write 0xa15b201 thru 0xa15bb05 (0x905 bytes) 2024-04-24T18:08:43.172 INFO:teuthology.orchestra.run.smithi049.stdout:4005 read 0xae2713c thru 0xae2b2a3 (0x4168 bytes) 2024-04-24T18:08:43.173 INFO:teuthology.orchestra.run.smithi049.stdout:4006 read 0x1b22ec7 thru 0x1b2b6f7 (0x8831 bytes) 2024-04-24T18:08:43.174 INFO:teuthology.orchestra.run.smithi049.stdout:4008 read 0x7bcfb60 thru 0x7bd39ae (0x3e4f bytes) 2024-04-24T18:08:43.175 INFO:teuthology.orchestra.run.smithi049.stdout:4009 write 0x4aa0551 thru 0x4aab0a4 (0xab54 bytes) 2024-04-24T18:08:43.179 INFO:teuthology.orchestra.run.smithi049.stdout:4010 read 0x232c09 thru 0x23d153 (0xa54b bytes) 2024-04-24T18:08:43.182 INFO:teuthology.orchestra.run.smithi049.stdout:4013 read 0xc6ceef0 thru 0xc6d18d3 (0x29e4 bytes) 2024-04-24T18:08:43.182 INFO:teuthology.orchestra.run.smithi049.stdout:4014 punch from 0xf6e8e0 to 0xf76650, (0x7d70 bytes) 2024-04-24T18:08:43.183 INFO:teuthology.orchestra.run.smithi049.stdout:4015 trunc from 0xe572332 to 0xd27f514 2024-04-24T18:08:43.188 INFO:teuthology.orchestra.run.smithi049.stdout:4016 write 0x47715c9 thru 0x477e6bd (0xd0f5 bytes) 2024-04-24T18:08:43.212 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:43.219+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:43.223 INFO:teuthology.orchestra.run.smithi049.stdout:4018 write 0x9be11cd thru 0x9be52c2 (0x40f6 bytes) 2024-04-24T18:08:43.229 INFO:teuthology.orchestra.run.smithi049.stdout:4019 write 0xbb4cdad thru 0xbb5b7c4 (0xea18 bytes) 2024-04-24T18:08:43.246 INFO:teuthology.orchestra.run.smithi049.stdout:4020 trunc from 0xd27f514 to 0x8bf42c2 2024-04-24T18:08:43.278 INFO:teuthology.orchestra.run.smithi049.stdout:4021 write 0x9e7fa53 thru 0x9e87837 (0x7de5 bytes) 2024-04-24T18:08:43.290 INFO:teuthology.orchestra.run.smithi049.stdout:4022 trunc from 0x9e87838 to 0x6db328f 2024-04-24T18:08:43.312 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:43.320+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:43.314 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:43.314 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:43.323 INFO:teuthology.orchestra.run.smithi049.stdout:4023 write 0x33486c4 thru 0x33533f5 (0xad32 bytes) 2024-04-24T18:08:43.326 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:43.382 INFO:teuthology.orchestra.run.smithi049.stdout:4024 write 0x76523ad thru 0x7654592 (0x21e6 bytes) 2024-04-24T18:08:43.386 INFO:teuthology.orchestra.run.smithi049.stdout:4025 punch from 0x4c6da07 to 0x4c75ac3, (0x80bc bytes) 2024-04-24T18:08:43.386 INFO:teuthology.orchestra.run.smithi049.stdout:4028 read 0x21d559f thru 0x21e2bc2 (0xd624 bytes) 2024-04-24T18:08:43.388 INFO:teuthology.orchestra.run.smithi049.stdout:4029 writesame 0x1cccd37 thru 0x1ccf53d data_size 0x1(0x2807 bytes) 2024-04-24T18:08:43.405 INFO:teuthology.orchestra.run.smithi049.stdout:4031 read 0x160761d thru 0x161635f (0xed43 bytes) 2024-04-24T18:08:43.408 INFO:teuthology.orchestra.run.smithi049.stdout:4032 read 0x6bfad10 thru 0x6c093fd (0xe6ee bytes) 2024-04-24T18:08:43.408 INFO:teuthology.orchestra.run.smithi049.stdout:4033 read 0x6108889 thru 0x611586d (0xcfe5 bytes) 2024-04-24T18:08:43.410 INFO:teuthology.orchestra.run.smithi049.stdout:4035 read 0x8d2906 thru 0x8d91ad (0x68a8 bytes) 2024-04-24T18:08:43.411 INFO:teuthology.orchestra.run.smithi049.stdout:4037 read 0x75d6636 thru 0x75da45d (0x3e28 bytes) 2024-04-24T18:08:43.412 INFO:teuthology.orchestra.run.smithi049.stdout:4038 read 0x5134b7c thru 0x5137b91 (0x3016 bytes) 2024-04-24T18:08:43.412 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:43.421+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:43.413 INFO:teuthology.orchestra.run.smithi049.stdout:4039 read 0x4a23645 thru 0x4a271bf (0x3b7b bytes) 2024-04-24T18:08:43.413 INFO:teuthology.orchestra.run.smithi049.stdout:4041 writesame 0x5bca666 thru 0x5bd40a5 data_size 0x2(0x9a40 bytes) 2024-04-24T18:08:43.446 INFO:teuthology.orchestra.run.smithi049.stdout:4043 read 0x5363beb thru 0x5372bd6 (0xefec bytes) 2024-04-24T18:08:43.449 INFO:teuthology.orchestra.run.smithi049.stdout:4044 trunc from 0x7654593 to 0x7e67361 2024-04-24T18:08:43.454 INFO:teuthology.orchestra.run.smithi049.stdout:4045 punch from 0x13d1977 to 0x13d6629, (0x4cb2 bytes) 2024-04-24T18:08:43.465 INFO:teuthology.orchestra.run.smithi049.stdout:4047 write 0xcf3db4e thru 0xcf4629c (0x874f bytes) 2024-04-24T18:08:43.474 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.484 INFO:teuthology.orchestra.run.smithi049.stdout:4049 trunc from 0xcf4629d to 0x4aa50a6 2024-04-24T18:08:43.490 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:43.490 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:43.502 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:43.516 INFO:teuthology.orchestra.run.smithi049.stdout:4050 write 0xbedfe05 thru 0xbee37f3 (0x39ef bytes) 2024-04-24T18:08:43.530 INFO:teuthology.orchestra.run.smithi049.stdout:4051 write 0xeb71f06 thru 0xeb72e87 (0xf82 bytes) 2024-04-24T18:08:43.533 INFO:teuthology.orchestra.run.smithi049.stdout:4052 write 0x78057f8 thru 0x78130b2 (0xd8bb bytes) 2024-04-24T18:08:43.542 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:43.551+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:43.544 INFO:teuthology.orchestra.run.smithi049.stdout:4055 punch from 0x905e130 to 0x9062c7a, (0x4b4a bytes) 2024-04-24T18:08:43.545 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.554 INFO:teuthology.orchestra.run.smithi049.stdout:4058 trunc from 0xeb72e88 to 0x6201712 2024-04-24T18:08:43.555 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:43.566 INFO:teuthology.orchestra.run.smithi049.stdout:4060 read 0xa7a6b8 thru 0xa7e0b5 (0x39fe bytes) 2024-04-24T18:08:43.568 INFO:teuthology.orchestra.run.smithi049.stdout:4063 punch from 0x1b641b7 to 0x1b6c6dd, (0x8526 bytes) 2024-04-24T18:08:43.568 INFO:teuthology.orchestra.run.smithi049.stdout:4066 read 0x26a4346 thru 0x26ad59b (0x9256 bytes) 2024-04-24T18:08:43.570 INFO:teuthology.orchestra.run.smithi049.stdout:4068 read 0x27fd001 thru 0x280528c (0x828c bytes) 2024-04-24T18:08:43.571 INFO:teuthology.orchestra.run.smithi049.stdout:4069 punch from 0x32dd261 to 0x32e0c8a, (0x3a29 bytes) 2024-04-24T18:08:43.571 INFO:teuthology.orchestra.run.smithi049.stdout:4070 write 0x3a0bbec thru 0x3a0c5be (0x9d3 bytes) 2024-04-24T18:08:43.579 INFO:teuthology.orchestra.run.smithi049.stdout:4071 trunc from 0x6201712 to 0x5a63614 2024-04-24T18:08:43.595 INFO:teuthology.orchestra.run.smithi049.stdout:4073 writesame 0xa0d7c05 thru 0xa0db3e8 data_size 0xe(0x37e4 bytes) 2024-04-24T18:08:43.595 INFO:teuthology.orchestra.run.smithi049.stdout:4075 read 0x992b78f thru 0x99336e2 (0x7f54 bytes) 2024-04-24T18:08:43.602 INFO:teuthology.orchestra.run.smithi049.stdout:4077 writesame 0xc89cedf thru 0xc8a792d data_size 0x1(0xaa4f bytes) 2024-04-24T18:08:43.609 INFO:teuthology.orchestra.run.smithi049.stdout:4078 writesame 0xdabd55c thru 0xdac2a77 data_size 0xd(0x551c bytes) 2024-04-24T18:08:43.609 INFO:teuthology.orchestra.run.smithi049.stdout:4079 punch from 0xc4c86e1 to 0xc4d586d, (0xd18c bytes) 2024-04-24T18:08:43.609 INFO:teuthology.orchestra.run.smithi049.stdout:4080 read 0x2b45366 thru 0x2b4c548 (0x71e3 bytes) 2024-04-24T18:08:43.611 INFO:teuthology.orchestra.run.smithi049.stdout:4082 punch from 0x76cbef3 to 0x76db7e5, (0xf8f2 bytes) 2024-04-24T18:08:43.611 INFO:teuthology.orchestra.run.smithi049.stdout:4083 write 0xd5c0ed7 thru 0xd5cbde8 (0xaf12 bytes) 2024-04-24T18:08:43.614 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:43.622+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:43.626 INFO:teuthology.orchestra.run.smithi049.stdout:4084 writesame 0x69b131a thru 0x69c0e1b data_size 0x59(0xfb02 bytes) 2024-04-24T18:08:43.640 INFO:teuthology.orchestra.run.smithi049.stdout:4085 punch from 0x5f856bb to 0x5f930a6, (0xd9eb bytes) 2024-04-24T18:08:43.645 INFO:teuthology.orchestra.run.smithi049.stdout:4087 trunc from 0xdac2a78 to 0x9bbd00b 2024-04-24T18:08:43.655 INFO:teuthology.orchestra.run.smithi049.stdout:4088 write 0x6af5703 thru 0x6afe3a9 (0x8ca7 bytes) 2024-04-24T18:08:43.659 INFO:teuthology.orchestra.run.smithi049.stdout:4090 read 0x1332aa0 thru 0x133ff1a (0xd47b bytes) 2024-04-24T18:08:43.662 INFO:teuthology.orchestra.run.smithi049.stdout:4091 read 0x93666ab thru 0x9370dab (0xa701 bytes) 2024-04-24T18:08:43.662 INFO:teuthology.orchestra.run.smithi049.stdout:4092 read 0x8169606 thru 0x8174502 (0xaefd bytes) 2024-04-24T18:08:43.662 INFO:teuthology.orchestra.run.smithi049.stdout:4093 read 0x688993c thru 0x6890d4c (0x7411 bytes) 2024-04-24T18:08:43.671 INFO:teuthology.orchestra.run.smithi049.stdout:4094 write 0xc63d52f thru 0xc6493ab (0xbe7d bytes) 2024-04-24T18:08:43.683 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:43.683 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:43.692 INFO:teuthology.orchestra.run.smithi049.stdout:4095 write 0xd63c6ab thru 0xd63d625 (0xf7b bytes) 2024-04-24T18:08:43.695 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:43.696 INFO:teuthology.orchestra.run.smithi049.stdout:4096 writesame 0xd422766 thru 0xd43073a data_size 0x1(0xdfd5 bytes) 2024-04-24T18:08:43.712 INFO:teuthology.orchestra.run.smithi049.stdout:4097 trunc from 0xd63d626 to 0xce1d4 2024-04-24T18:08:43.716 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:43.723+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:43.731 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:43.771 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:43.790 INFO:teuthology.orchestra.run.smithi049.stdout:4098 write 0x8990e1 thru 0x89d6ac (0x45cc bytes) 2024-04-24T18:08:43.794 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:43.797 INFO:teuthology.orchestra.run.smithi049.stdout:4099 punch from 0x4535f7 to 0x461119, (0xdb22 bytes) 2024-04-24T18:08:43.797 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.797 INFO:teuthology.orchestra.run.smithi049.stdout:4101 read 0x6fed0f thru 0x70847b (0x976d bytes) 2024-04-24T18:08:43.797 INFO:teuthology.orchestra.run.smithi049.stdout:4103 read 0x6efccb thru 0x6fa82d (0xab63 bytes) 2024-04-24T18:08:43.806 INFO:teuthology.orchestra.run.smithi049.stdout:4107 trunc from 0x89d6ad to 0x7e10608 2024-04-24T18:08:43.811 INFO:teuthology.orchestra.run.smithi049.stdout:4108 read 0x446aff4 thru 0x447a556 (0xf563 bytes) 2024-04-24T18:08:43.812 INFO:teuthology.orchestra.run.smithi049.stdout:4109 write 0x63bf0db thru 0x63bfce4 (0xc0a bytes) 2024-04-24T18:08:43.816 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:43.824+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:43.817 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:43.825 INFO:teuthology.orchestra.run.smithi049.stdout:4110 write 0xd71bd3f thru 0xd723092 (0x7354 bytes) 2024-04-24T18:08:43.839 INFO:teuthology.orchestra.run.smithi049.stdout:4114 read 0x5bf66a5 thru 0x5bffad6 (0x9432 bytes) 2024-04-24T18:08:43.839 INFO:teuthology.orchestra.run.smithi049.stdout:4115 punch from 0x7d034ef to 0x7d0fa6c, (0xc57d bytes) 2024-04-24T18:08:43.839 INFO:teuthology.orchestra.run.smithi049.stdout:4116 punch from 0xcc2c2ec to 0xcc31143, (0x4e57 bytes) 2024-04-24T18:08:43.840 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:43.842 INFO:teuthology.orchestra.run.smithi049.stdout:4118 trunc from 0xd723093 to 0xa9f4802 2024-04-24T18:08:43.857 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.857 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.857 INFO:teuthology.orchestra.run.smithi049.stdout:4122 writesame 0x45fad12 thru 0x4601223 data_size 0x11(0x6512 bytes) 2024-04-24T18:08:43.863 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:43.872 INFO:teuthology.orchestra.run.smithi049.stdout:4123 writesame 0xb32eaab thru 0xb3342eb data_size 0x11(0x5841 bytes) 2024-04-24T18:08:43.872 INFO:teuthology.orchestra.run.smithi049.stdout:4124 read 0x289d263 thru 0x289fa7a (0x2818 bytes) 2024-04-24T18:08:43.872 INFO:teuthology.orchestra.run.smithi049.stdout:4125 write 0x5cd06a3 thru 0x5cd56d2 (0x5030 bytes) 2024-04-24T18:08:43.874 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:43.874 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:43.874 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:43.878 INFO:teuthology.orchestra.run.smithi049.stdout:4127 write 0x11c605a thru 0x11d0c0b (0xabb2 bytes) 2024-04-24T18:08:43.887 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:43.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:43.888 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:43.890 INFO:teuthology.orchestra.run.smithi049.stdout:4128 read 0xa9a15dc thru 0xa9a554a (0x3f6f bytes) 2024-04-24T18:08:43.890 INFO:teuthology.orchestra.run.smithi049.stdout:4133 write 0x591447c thru 0x591e550 (0xa0d5 bytes) 2024-04-24T18:08:43.891 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:43.891 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:43.905 INFO:teuthology.orchestra.run.smithi049.stdout:4134 read 0x89b2bc thru 0x89e134 (0x2e79 bytes) 2024-04-24T18:08:43.906 INFO:teuthology.orchestra.run.smithi049.stdout:4136 write 0xb0c801d thru 0xb0c8335 (0x319 bytes) 2024-04-24T18:08:43.909 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.909 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:43.909 INFO:teuthology.orchestra.run.smithi049.stdout:4143 punch from 0x6f2e19 to 0x6f4cda, (0x1ec1 bytes) 2024-04-24T18:08:43.909 INFO:teuthology.orchestra.run.smithi049.stdout:4144 read 0x3e1eafd thru 0x3e216da (0x2bde bytes) 2024-04-24T18:08:43.909 INFO:teuthology.orchestra.run.smithi049.stdout:4146 write 0x931a32c thru 0x93266ec (0xc3c1 bytes) 2024-04-24T18:08:43.917 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:43.924+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:43.920 INFO:teuthology.orchestra.run.smithi049.stdout:4149 trunc from 0xb3342ec to 0xb76eb5d 2024-04-24T18:08:43.925 INFO:teuthology.orchestra.run.smithi049.stdout:4150 read 0x1a0c9e thru 0x1aa1fa (0x955d bytes) 2024-04-24T18:08:43.925 INFO:teuthology.orchestra.run.smithi049.stdout:4151 punch from 0x14dba5 to 0x157112, (0x956d bytes) 2024-04-24T18:08:43.925 INFO:teuthology.orchestra.run.smithi049.stdout:4152 read 0x93d104e thru 0x93df660 (0xe613 bytes) 2024-04-24T18:08:43.927 INFO:teuthology.orchestra.run.smithi049.stdout:4153 trunc from 0xb76eb5d to 0xc69f6b6 2024-04-24T18:08:43.932 INFO:teuthology.orchestra.run.smithi049.stdout:4154 writesame 0x65dd2fb thru 0x65ebe44 data_size 0x3(0xeb4a bytes) 2024-04-24T18:08:43.944 INFO:teuthology.orchestra.run.smithi049.stdout:4155 punch from 0x9bf5248 to 0x9bfdea4, (0x8c5c bytes) 2024-04-24T18:08:43.953 INFO:teuthology.orchestra.run.smithi049.stdout:4156 write 0xec0c070 thru 0xec15844 (0x97d5 bytes) 2024-04-24T18:08:43.965 INFO:teuthology.orchestra.run.smithi049.stdout:4157 trunc from 0xec15845 to 0xcb4dada 2024-04-24T18:08:43.975 INFO:teuthology.orchestra.run.smithi049.stdout:4158 read 0x45fb49f thru 0x4608703 (0xd265 bytes) 2024-04-24T18:08:43.976 INFO:teuthology.orchestra.run.smithi049.stdout:4159 read 0x82c16d9 thru 0x82c4ef6 (0x381e bytes) 2024-04-24T18:08:43.976 INFO:teuthology.orchestra.run.smithi049.stdout:4160 read 0xacbc034 thru 0xacc1967 (0x5934 bytes) 2024-04-24T18:08:43.976 INFO:teuthology.orchestra.run.smithi049.stdout:4161 write 0x9860f9c thru 0x9868223 (0x7288 bytes) 2024-04-24T18:08:43.987 INFO:teuthology.orchestra.run.smithi049.stdout:4163 read 0x9adb11e thru 0x9ae3c26 (0x8b09 bytes) 2024-04-24T18:08:43.987 INFO:teuthology.orchestra.run.smithi049.stdout:4164 read 0xd93ad0 thru 0xd9a763 (0x6c94 bytes) 2024-04-24T18:08:43.987 INFO:teuthology.orchestra.run.smithi049.stdout:4166 write 0x1b89442 thru 0x1b899c6 (0x585 bytes) 2024-04-24T18:08:43.989 INFO:teuthology.orchestra.run.smithi049.stdout:4167 write 0x7637c8d thru 0x763ee5e (0x71d2 bytes) 2024-04-24T18:08:44.000 INFO:teuthology.orchestra.run.smithi049.stdout:4168 write 0xb5db2ec thru 0xb5e7a15 (0xc72a bytes) 2024-04-24T18:08:44.015 INFO:teuthology.orchestra.run.smithi049.stdout:4169 writesame 0x97bd16c thru 0x97c11d3 data_size 0x24(0x4068 bytes) 2024-04-24T18:08:44.017 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:44.025+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:44.018 INFO:teuthology.orchestra.run.smithi049.stdout:4170 read 0xcb1a20 thru 0xcbb478 (0x9a59 bytes) 2024-04-24T18:08:44.019 INFO:teuthology.orchestra.run.smithi049.stdout:4171 trunc from 0xcb4dada to 0xda71bcc 2024-04-24T18:08:44.024 INFO:teuthology.orchestra.run.smithi049.stdout:4172 writesame 0x47c4dd2 thru 0x47cf173 data_size 0x47(0xa3a2 bytes) 2024-04-24T18:08:44.030 INFO:teuthology.orchestra.run.smithi049.stdout:4173 punch from 0xd773c44 to 0xd783559, (0xf915 bytes) 2024-04-24T18:08:44.030 INFO:teuthology.orchestra.run.smithi049.stdout:4174 punch from 0x522c1c5 to 0x523a149, (0xdf84 bytes) 2024-04-24T18:08:44.030 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.030 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.038 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:44.039 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:44.039 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:44.043 INFO:teuthology.orchestra.run.smithi049.stdout:4177 trunc from 0xda71bcc to 0x1a635a4 2024-04-24T18:08:44.056 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:44.087 INFO:teuthology.orchestra.run.smithi049.stdout:4178 writesame 0xda37f9e thru 0xda402bb data_size 0xd(0x831e bytes) 2024-04-24T18:08:44.087 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.087 INFO:teuthology.orchestra.run.smithi049.stdout:4181 read 0xd53ad91 thru 0xd547ef4 (0xd164 bytes) 2024-04-24T18:08:44.087 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.100 INFO:teuthology.orchestra.run.smithi049.stdout:4183 trunc from 0xda402bc to 0x17df036 2024-04-24T18:08:44.106 INFO:teuthology.orchestra.run.smithi049.stdout:4184 punch from 0x133ae3 to 0x13fd8d, (0xc2aa bytes) 2024-04-24T18:08:44.106 INFO:teuthology.orchestra.run.smithi049.stdout:4186 read 0x10a5bc thru 0x10f3bd (0x4e02 bytes) 2024-04-24T18:08:44.106 INFO:teuthology.orchestra.run.smithi049.stdout:4187 punch from 0x891cc0 to 0x894a70, (0x2db0 bytes) 2024-04-24T18:08:44.118 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:44.127+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:44.125 INFO:teuthology.orchestra.run.smithi049.stdout:4188 write 0xd2c4e0d thru 0xd2c5b10 (0xd04 bytes) 2024-04-24T18:08:44.129 INFO:teuthology.orchestra.run.smithi049.stdout:4189 read 0x14c7b1f thru 0x14cf594 (0x7a76 bytes) 2024-04-24T18:08:44.129 INFO:teuthology.orchestra.run.smithi049.stdout:4190 read 0x99dfe5f thru 0x99ee329 (0xe4cb bytes) 2024-04-24T18:08:44.129 INFO:teuthology.orchestra.run.smithi049.stdout:4191 writesame 0x4e89e8a thru 0x4e9672e data_size 0x1(0xc8a5 bytes) 2024-04-24T18:08:44.146 INFO:teuthology.orchestra.run.smithi049.stdout:4192 write 0x409c684 thru 0x40a1379 (0x4cf6 bytes) 2024-04-24T18:08:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:4193 punch from 0x95207c5 to 0x952f659, (0xee94 bytes) 2024-04-24T18:08:44.154 INFO:teuthology.orchestra.run.smithi049.stdout:4196 write 0x520354c thru 0x520eea7 (0xb95c bytes) 2024-04-24T18:08:44.173 INFO:teuthology.orchestra.run.smithi049.stdout:4198 writesame 0xd36d7e0 thru 0xd375f24 data_size 0x33(0x8745 bytes) 2024-04-24T18:08:44.175 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.175 INFO:teuthology.orchestra.run.smithi049.stdout:4201 flatten 2024-04-24T18:08:44.192 INFO:teuthology.orchestra.run.smithi049.stdout:4202 trunc from 0xd375f25 to 0x3df22d4 2024-04-24T18:08:44.209 INFO:teuthology.orchestra.run.smithi049.stdout:4203 write 0x1300b10 thru 0x130609d (0x558e bytes) 2024-04-24T18:08:44.216 INFO:teuthology.orchestra.run.smithi049.stdout:4204 read 0x2d32b96 thru 0x2d3c1c0 (0x962b bytes) 2024-04-24T18:08:44.217 INFO:teuthology.orchestra.run.smithi049.stdout:4205 read 0x26135de thru 0x2617ecf (0x48f2 bytes) 2024-04-24T18:08:44.219 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:44.227+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:44.220 INFO:teuthology.orchestra.run.smithi049.stdout:4206 trunc from 0x3df22d4 to 0xdb36f1 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.547608623999999, 2024-04-24T18:08:44.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:08:44.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:08:44.252 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.635133818, 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:08:44.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:08:44.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:08:44.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.207665295, 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:08:44.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.261 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2076440099999992, 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:08:44.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:08:44.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:08:44.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.265 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2076266469999997, 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:44.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:08:44.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:08:44.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2075812540000008, 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:08:44.269 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:08:44.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:08:44.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2075537529999991, 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.207535193, 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:44.280 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2075113050000006, 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2074934109999997, 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.284 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.287 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2074587690000005, 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2073642410000005, 2024-04-24T18:08:44.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2073455600000003, 2024-04-24T18:08:44.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.207290081, 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2072556950000006, 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2072337359999992, 2024-04-24T18:08:44.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:08:44.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2072166529999997, 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.310 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2072001930000003, 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:08:44.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2071262780000005, 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:08:44.317 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:08:44.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:08:44.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:08:44.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 8.2070737549999997, 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.321 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.324 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:44.325 INFO:teuthology.orchestra.run.smithi049.stdout:4208 writesame 0xd535e2a thru 0xd538bdf data_size 0x1(0x2db6 bytes) 2024-04-24T18:08:44.325 INFO:teuthology.orchestra.run.smithi049.stdout:4209 write 0x796d78d thru 0x7973623 (0x5e97 bytes) 2024-04-24T18:08:44.326 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:44.328 INFO:teuthology.orchestra.run.smithi049.stdout:4210 trunc from 0xd538be0 to 0x6f294a9 2024-04-24T18:08:44.328 INFO:teuthology.orchestra.run.smithi049.stdout:4212 punch from 0x4fe851d to 0x4ff69e4, (0xe4c7 bytes) 2024-04-24T18:08:44.328 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.328 INFO:teuthology.orchestra.run.smithi049.stdout:4216 read 0xf8d23 thru 0xf94e5 (0x7c3 bytes) 2024-04-24T18:08:44.328 INFO:teuthology.orchestra.run.smithi049.stdout:4217 trunc from 0x6f294a9 to 0xe9c69af 2024-04-24T18:08:44.328 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4222 write 0xe52409f thru 0xe52d0c8 (0x902a bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4224 read 0x1e0bddc thru 0x1e1795c (0xbb81 bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4228 read 0x61b9de0 thru 0x61c7ae9 (0xdd0a bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4230 read 0x384461b thru 0x3848b09 (0x44ef bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4231 write 0x9c34b8f thru 0x9c37156 (0x25c8 bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4234 punch from 0xdc6a083 to 0xdc6e5ec, (0x4569 bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4235 read 0x95e6fe7 thru 0x95ef74f (0x8769 bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4236 punch from 0xd418393 to 0xd418df2, (0xa5f bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4238 write 0xb2ab4f thru 0xb3a22d (0xf6df bytes) 2024-04-24T18:08:44.329 INFO:teuthology.orchestra.run.smithi049.stdout:4239 clone 43 order 21 su 16384 sc 2 2024-04-24T18:08:44.330 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:44.333+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:44.426 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:44.434+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:44.495 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:44.496 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:44.496 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:44.506 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:44.527 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:44.536+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:44.628 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:44.636+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:44.677 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:44.677 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:44.677 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:44.687 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:44.728 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:44.737+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:44.829 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:44.837+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:44.851 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396460404999999, 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:08:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:08:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396442200999999, 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396420786, 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:08:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:08:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:08:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396403634999999, 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:08:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396387458, 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:08:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:08:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396369621000002, 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:08:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396339017999999, 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:08:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396318859000001, 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:08:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396302582000001, 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396284991000002, 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:08:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396269122, 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396250576, 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:08:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396234532000001, 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:08:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396217239999999, 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:08:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396195584000001, 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:08:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396176753999999, 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396160419000001, 2024-04-24T18:08:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:08:44.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:08:44.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396144593999999, 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.929 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.932 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396126169999999, 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:08:44.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:08:44.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:44.936 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 18.396109795000001, 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:44.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:44.940 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:44.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:44.942 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:44.949+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:45.041 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:45.050+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:45.083 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #41, image image_client.0-clone41 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent42 2024-04-24T18:08:45.138 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:45.138 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:45.138 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:45.138 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:45.142 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:45.151+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:45.152 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:45.202 INFO:teuthology.orchestra.run.smithi049.stdout:4240 read 0x4444bbf thru 0x4448c2f (0x4071 bytes) 2024-04-24T18:08:45.204 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.204 INFO:teuthology.orchestra.run.smithi049.stdout:4242 write 0x4e08256 thru 0x4e12655 (0xa400 bytes) 2024-04-24T18:08:45.214 INFO:teuthology.orchestra.run.smithi049.stdout:4243 read 0x699cd2d thru 0x69a2877 (0x5b4b bytes) 2024-04-24T18:08:45.215 INFO:teuthology.orchestra.run.smithi049.stdout:4245 writesame 0xc7dd31d thru 0xc7ec319 data_size 0x1(0xeffd bytes) 2024-04-24T18:08:45.235 INFO:teuthology.orchestra.run.smithi049.stdout:4248 writesame 0x1e6e5d5 thru 0x1e76e13 data_size 0x1(0x883f bytes) 2024-04-24T18:08:45.244 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:45.252+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:45.262 INFO:teuthology.orchestra.run.smithi049.stdout:4250 trunc from 0xe9c69af to 0x3a0d7ea 2024-04-24T18:08:45.279 INFO:teuthology.orchestra.run.smithi049.stdout:4252 read 0xfc6f4c thru 0xfcd7d8 (0x688d bytes) 2024-04-24T18:08:45.279 INFO:teuthology.orchestra.run.smithi049.stdout:4253 punch from 0x1fe3c0e to 0x1fe9cfd, (0x60ef bytes) 2024-04-24T18:08:45.290 INFO:teuthology.orchestra.run.smithi049.stdout:4254 trunc from 0x3a0d7ea to 0xbb03df6 2024-04-24T18:08:45.303 INFO:teuthology.orchestra.run.smithi049.stdout:4255 trunc from 0xbb03df6 to 0x50cefee 2024-04-24T18:08:45.305 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:45.305 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:45.305 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:45.305 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:45.305 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:45.309 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.309 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.318 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:45.319 INFO:teuthology.orchestra.run.smithi049.stdout:4258 writesame 0x816dd31 thru 0x8177b9d data_size 0xb(0x9e6d bytes) 2024-04-24T18:08:45.319 INFO:teuthology.orchestra.run.smithi049.stdout:4259 writesame 0x80adb09 thru 0x80afaed data_size 0x17(0x1fe5 bytes) 2024-04-24T18:08:45.322 INFO:teuthology.orchestra.run.smithi049.stdout:4260 punch from 0xe8dfa2 to 0xe92017, (0x4075 bytes) 2024-04-24T18:08:45.322 INFO:teuthology.orchestra.run.smithi049.stdout:4262 punch from 0x1d7957a to 0x1d7e426, (0x4eac bytes) 2024-04-24T18:08:45.322 INFO:teuthology.orchestra.run.smithi049.stdout:4264 punch from 0x4e333fc to 0x4e3acfe, (0x7902 bytes) 2024-04-24T18:08:45.333 INFO:teuthology.orchestra.run.smithi049.stdout:4267 write 0xcb6f635 thru 0xcb7945f (0x9e2b bytes) 2024-04-24T18:08:45.338 INFO:teuthology.orchestra.run.smithi049.stdout:4268 write 0xa5d2f70 thru 0xa5e06cd (0xd75e bytes) 2024-04-24T18:08:45.342 INFO:teuthology.orchestra.run.smithi049.stdout:4273 read 0xa0b3860 thru 0xa0bbe1d (0x85be bytes) 2024-04-24T18:08:45.342 INFO:teuthology.orchestra.run.smithi049.stdout:4274 write 0x4eb66b4 thru 0x4ebc977 (0x62c4 bytes) 2024-04-24T18:08:45.345 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:45.353+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:45.346 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.346 INFO:teuthology.orchestra.run.smithi049.stdout:4276 write 0x83c77ff thru 0x83d6cd7 (0xf4d9 bytes) 2024-04-24T18:08:45.349 INFO:teuthology.orchestra.run.smithi049.stdout:4278 read 0xbc67a08 thru 0xbc70950 (0x8f49 bytes) 2024-04-24T18:08:45.350 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.350 INFO:teuthology.orchestra.run.smithi049.stdout:4280 write 0xf0de53 thru 0xf140ed (0x629b bytes) 2024-04-24T18:08:45.356 INFO:teuthology.orchestra.run.smithi049.stdout:4282 read 0xa97bf9d thru 0xa98794b (0xb9af bytes) 2024-04-24T18:08:45.356 INFO:teuthology.orchestra.run.smithi049.stdout:4283 read 0x9e79a86 thru 0x9e80fc5 (0x7540 bytes) 2024-04-24T18:08:45.356 INFO:teuthology.orchestra.run.smithi049.stdout:4287 read 0x9976446 thru 0x997ad14 (0x48cf bytes) 2024-04-24T18:08:45.356 INFO:teuthology.orchestra.run.smithi049.stdout:4290 read 0x122e4c4 thru 0x12371d0 (0x8d0d bytes) 2024-04-24T18:08:45.356 INFO:teuthology.orchestra.run.smithi049.stdout:4291 read 0x223fe6c thru 0x224c55e (0xc6f3 bytes) 2024-04-24T18:08:45.356 INFO:teuthology.orchestra.run.smithi049.stdout:4292 writesame 0x7ff43fb thru 0x7ffd7eb data_size 0xb(0x93f1 bytes) 2024-04-24T18:08:45.361 INFO:teuthology.orchestra.run.smithi049.stdout:4293 write 0xa665dcc thru 0xa6723da (0xc60f bytes) 2024-04-24T18:08:45.372 INFO:teuthology.orchestra.run.smithi049.stdout:4294 trunc from 0xcb79460 to 0x5617386 2024-04-24T18:08:45.383 INFO:teuthology.orchestra.run.smithi049.stdout:4295 read 0x3396f9b thru 0x33a35bf (0xc625 bytes) 2024-04-24T18:08:45.388 INFO:teuthology.orchestra.run.smithi049.stdout:4296 trunc from 0x5617386 to 0x9bac27d 2024-04-24T18:08:45.393 INFO:teuthology.orchestra.run.smithi049.stdout:4297 write 0x584e337 thru 0x5857708 (0x93d2 bytes) 2024-04-24T18:08:45.407 INFO:teuthology.orchestra.run.smithi049.stdout:4298 writesame 0xd33b6c1 thru 0xd34339f data_size 0x1(0x7cdf bytes) 2024-04-24T18:08:45.408 INFO:teuthology.orchestra.run.smithi049.stdout:4300 read 0x88106a9 thru 0x88197dc (0x9134 bytes) 2024-04-24T18:08:45.408 INFO:teuthology.orchestra.run.smithi049.stdout:4303 writesame 0x319f1a4 thru 0x31adf69 data_size 0xf(0xedc6 bytes) 2024-04-24T18:08:45.414 INFO:teuthology.orchestra.run.smithi049.stdout:4304 punch from 0xbaa1b78 to 0xbaac1ca, (0xa652 bytes) 2024-04-24T18:08:45.414 INFO:teuthology.orchestra.run.smithi049.stdout:4305 punch from 0x9e312d1 to 0x9e31364, (0x93 bytes) 2024-04-24T18:08:45.415 INFO:teuthology.orchestra.run.smithi049.stdout:4306 write 0x66dbe4 thru 0x673676 (0x5a93 bytes) 2024-04-24T18:08:45.432 INFO:teuthology.orchestra.run.smithi049.stdout:4309 writesame 0xacde696 thru 0xace407c data_size 0x1(0x59e7 bytes) 2024-04-24T18:08:45.442 INFO:teuthology.orchestra.run.smithi049.stdout:4310 writesame 0x35d5180 thru 0x35df05f data_size 0x7c(0x9ee0 bytes) 2024-04-24T18:08:45.446 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:45.454+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:45.449 INFO:teuthology.orchestra.run.smithi049.stdout:4311 writesame 0x2532d0b thru 0x253f67f data_size 0x1(0xc975 bytes) 2024-04-24T18:08:45.470 INFO:teuthology.orchestra.run.smithi049.stdout:4313 trunc from 0xd3433a0 to 0x874cb36 2024-04-24T18:08:45.479 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349658448, 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349643907999997, 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:08:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:08:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349629057000001, 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349613908000002, 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349599122000001, 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:08:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:08:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349584532999998, 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:08:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:08:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349561708000003, 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349546054000001, 2024-04-24T18:08:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:08:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349530932, 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:08:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:08:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349515115999999, 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349500114000001, 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:08:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:08:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349485250000001, 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:08:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:08:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349464132999998, 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:08:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349448903000003, 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:08:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:08:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:08:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:08:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.349434166000002, 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.344332956000002, 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.344297109999999, 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.545 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.344277103000003, 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.549 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.342455911999998, 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.553 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.342430632000003, 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:45.557 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:45.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:45.561 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:45.562 INFO:teuthology.orchestra.run.smithi049.stdout:4314 read 0x6fb276d thru 0x6fb6e26 (0x46ba bytes) 2024-04-24T18:08:45.562 INFO:teuthology.orchestra.run.smithi049.stdout:4315 write 0xf9e9d1 thru 0xfa38ba (0x4eea bytes) 2024-04-24T18:08:45.562 INFO:teuthology.orchestra.run.smithi049.stdout:4316 write 0x1b2f352 thru 0x1b379f2 (0x86a1 bytes) 2024-04-24T18:08:45.563 INFO:teuthology.orchestra.run.smithi049.stdout:4318 read 0x35e1ee3 thru 0x35f1783 (0xf8a1 bytes) 2024-04-24T18:08:45.563 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.564 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:45.565 INFO:teuthology.orchestra.run.smithi049.stdout:4321 write 0xdfdfcc0 thru 0xdfed55e (0xd89f bytes) 2024-04-24T18:08:45.565 INFO:teuthology.orchestra.run.smithi049.stdout:4322 read 0xd9f5895 thru 0xd9fb85f (0x5fcb bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4324 trunc from 0xdfed55f to 0xbf88078 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4325 write 0x8f29f3b thru 0x8f3631b (0xc3e1 bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4326 write 0x887e041 thru 0x8882eca (0x4e8a bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4328 read 0x2c38725 thru 0x2c44024 (0xb900 bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4329 writesame 0x12a4b12 thru 0x12aae7b data_size 0x19(0x636a bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4331 write 0x335c341 thru 0x335c38d (0x4d bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4332 read 0xaceab8c thru 0xaceed85 (0x41fa bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4333 punch from 0xceae1f to 0xcf5e83, (0xb064 bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4336 read 0xaaf1bed thru 0xaaf28cd (0xce1 bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4337 read 0xad2dc5d thru 0xad3a67d (0xca21 bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4339 writesame 0xb176cb1 thru 0xb17b9b9 data_size 0x25(0x4d09 bytes) 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.566 INFO:teuthology.orchestra.run.smithi049.stdout:4341 writesame 0xab5151a thru 0xab5b4c0 data_size 0x1(0x9fa7 bytes) 2024-04-24T18:08:45.567 INFO:teuthology.orchestra.run.smithi049.stdout:4342 punch from 0xbea8e37 to 0xbeb6d31, (0xdefa bytes) 2024-04-24T18:08:45.567 INFO:teuthology.orchestra.run.smithi049.stdout:4343 writesame 0x960ee56 thru 0x9618492 data_size 0x1(0x963d bytes) 2024-04-24T18:08:45.567 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:45.570+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:45.568 INFO:teuthology.orchestra.run.smithi049.stdout:4345 read 0x9a4b95f thru 0x9a53e59 (0x84fb bytes) 2024-04-24T18:08:45.568 INFO:teuthology.orchestra.run.smithi049.stdout:4346 writesame 0xa4ce1d2 thru 0xa4dbaee data_size 0x61(0xd91d bytes) 2024-04-24T18:08:45.572 INFO:teuthology.orchestra.run.smithi049.stdout:4348 punch from 0x9982c to 0x9d5cd, (0x3da1 bytes) 2024-04-24T18:08:45.578 INFO:teuthology.orchestra.run.smithi049.stdout:4349 trunc from 0xbf88078 to 0x721cb1b 2024-04-24T18:08:45.590 INFO:teuthology.orchestra.run.smithi049.stdout:4350 punch from 0x1d19e4b to 0x1d1d7c0, (0x3975 bytes) 2024-04-24T18:08:45.598 INFO:teuthology.orchestra.run.smithi049.stdout:4351 write 0x856f772 thru 0x856f845 (0xd4 bytes) 2024-04-24T18:08:45.611 INFO:teuthology.orchestra.run.smithi049.stdout:4352 write 0xc9609ce thru 0xc963f2f (0x3562 bytes) 2024-04-24T18:08:45.614 INFO:teuthology.orchestra.run.smithi049.stdout:4353 writesame 0x59a3a86 thru 0x59a6f62 data_size 0xd(0x34dd bytes) 2024-04-24T18:08:45.626 INFO:teuthology.orchestra.run.smithi049.stdout:4354 trunc from 0xc963f30 to 0x33f47fd 2024-04-24T18:08:45.639 INFO:teuthology.orchestra.run.smithi049.stdout:4355 read 0x59b3ac thru 0x5a3a05 (0x865a bytes) 2024-04-24T18:08:45.645 INFO:teuthology.orchestra.run.smithi049.stdout:4356 writesame 0x3b61a00 thru 0x3b6c2ea data_size 0x1(0xa8eb bytes) 2024-04-24T18:08:45.645 INFO:teuthology.orchestra.run.smithi049.stdout:4357 read 0x996566 thru 0x99c396 (0x5e31 bytes) 2024-04-24T18:08:45.646 INFO:teuthology.orchestra.run.smithi049.stdout:4360 read 0x1448711 thru 0x144fdd5 (0x76c5 bytes) 2024-04-24T18:08:45.646 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.660 INFO:teuthology.orchestra.run.smithi049.stdout:4362 write 0x99bcd1f thru 0x99cabe2 (0xdec4 bytes) 2024-04-24T18:08:45.663 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:45.670+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:45.663 INFO:teuthology.orchestra.run.smithi049.stdout:4363 read 0x1347ff0 thru 0x134a0f5 (0x2106 bytes) 2024-04-24T18:08:45.664 INFO:teuthology.orchestra.run.smithi049.stdout:4364 read 0x2eb884b thru 0x2ec098f (0x8145 bytes) 2024-04-24T18:08:45.664 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.664 INFO:teuthology.orchestra.run.smithi049.stdout:4367 write 0x7da341e thru 0x7da976d (0x6350 bytes) 2024-04-24T18:08:45.668 INFO:teuthology.orchestra.run.smithi049.stdout:4368 read 0x278ce84 thru 0x2791b5f (0x4cdc bytes) 2024-04-24T18:08:45.668 INFO:teuthology.orchestra.run.smithi049.stdout:4370 punch from 0x6248ca5 to 0x62500af, (0x740a bytes) 2024-04-24T18:08:45.673 INFO:teuthology.orchestra.run.smithi049.stdout:4371 trunc from 0x99cabe3 to 0x4fa2995 2024-04-24T18:08:45.683 INFO:teuthology.orchestra.run.smithi049.stdout:4373 punch from 0xfdd156 to 0xfe22e9, (0x5193 bytes) 2024-04-24T18:08:45.683 INFO:teuthology.orchestra.run.smithi049.stdout:4375 read 0x3240f33 thru 0x3249bc8 (0x8c96 bytes) 2024-04-24T18:08:45.699 INFO:teuthology.orchestra.run.smithi049.stdout:4376 writesame 0xd330bbc thru 0xd33b22e data_size 0x1(0xa673 bytes) 2024-04-24T18:08:45.699 INFO:teuthology.orchestra.run.smithi049.stdout:4377 write 0xc092823 thru 0xc09de74 (0xb652 bytes) 2024-04-24T18:08:45.705 INFO:teuthology.orchestra.run.smithi049.stdout:4379 trunc from 0xd33b22f to 0xee1e142 2024-04-24T18:08:45.710 INFO:teuthology.orchestra.run.smithi049.stdout:4381 write 0xe9d2661 thru 0xe9dd731 (0xb0d1 bytes) 2024-04-24T18:08:45.714 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.727 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:45.727 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:45.727 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:45.727 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:45.729 INFO:teuthology.orchestra.run.smithi049.stdout:4384 trunc from 0xee1e142 to 0x1266cda 2024-04-24T18:08:45.742 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:45.743 INFO:teuthology.orchestra.run.smithi049.stdout:4385 read 0xa02ccb thru 0xa05065 (0x239b bytes) 2024-04-24T18:08:45.743 INFO:teuthology.orchestra.run.smithi049.stdout:4387 read 0x9bbdac thru 0x9bf84c (0x3aa1 bytes) 2024-04-24T18:08:45.744 INFO:teuthology.orchestra.run.smithi049.stdout:4389 read 0x123d787 thru 0x123f821 (0x209b bytes) 2024-04-24T18:08:45.750 INFO:teuthology.orchestra.run.smithi049.stdout:4390 trunc from 0x1266cda to 0x62e23f0 2024-04-24T18:08:45.756 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.757 INFO:teuthology.orchestra.run.smithi049.stdout:4393 write 0x449b49c thru 0x44a61e2 (0xad47 bytes) 2024-04-24T18:08:45.764 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:45.772+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:45.770 INFO:teuthology.orchestra.run.smithi049.stdout:4394 write 0x8f0fc05 thru 0x8f1a9b6 (0xadb2 bytes) 2024-04-24T18:08:45.774 INFO:teuthology.orchestra.run.smithi049.stdout:4395 write 0x1d56d61 thru 0x1d64e91 (0xe131 bytes) 2024-04-24T18:08:45.785 INFO:teuthology.orchestra.run.smithi049.stdout:4396 write 0xa867c18 thru 0xa868c3a (0x1023 bytes) 2024-04-24T18:08:45.789 INFO:teuthology.orchestra.run.smithi049.stdout:4397 read 0x9083a83 thru 0x908bb7f (0x80fd bytes) 2024-04-24T18:08:45.789 INFO:teuthology.orchestra.run.smithi049.stdout:4401 read 0x1b67aa7 thru 0x1b714e9 (0x9a43 bytes) 2024-04-24T18:08:45.796 INFO:teuthology.orchestra.run.smithi049.stdout:4402 write 0xbdae747 thru 0xbdbad5f (0xc619 bytes) 2024-04-24T18:08:45.806 INFO:teuthology.orchestra.run.smithi049.stdout:4405 writesame 0xc7faa64 thru 0xc804a96 data_size 0x1(0xa033 bytes) 2024-04-24T18:08:45.806 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.813 INFO:teuthology.orchestra.run.smithi049.stdout:4407 trunc from 0xc804a97 to 0x6234f98 2024-04-24T18:08:45.824 INFO:teuthology.orchestra.run.smithi049.stdout:4408 trunc from 0x6234f98 to 0x6540211 2024-04-24T18:08:45.829 INFO:teuthology.orchestra.run.smithi049.stdout:4409 read 0x4915270 thru 0x4915bb6 (0x947 bytes) 2024-04-24T18:08:45.829 INFO:teuthology.orchestra.run.smithi049.stdout:4410 write 0x21bdbe9 thru 0x21c72d6 (0x96ee bytes) 2024-04-24T18:08:45.833 INFO:teuthology.orchestra.run.smithi049.stdout:4411 read 0x505c062 thru 0x506b2fe (0xf29d bytes) 2024-04-24T18:08:45.833 INFO:teuthology.orchestra.run.smithi049.stdout:4412 read 0x5eab0a1 thru 0x5ead9c2 (0x2922 bytes) 2024-04-24T18:08:45.833 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.841 INFO:teuthology.orchestra.run.smithi049.stdout:4414 trunc from 0x6540211 to 0xd0f595e 2024-04-24T18:08:45.848 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:08:45.851 INFO:teuthology.orchestra.run.smithi049.stdout:4415 trunc from 0xd0f595e to 0xd216243 2024-04-24T18:08:45.857 INFO:teuthology.orchestra.run.smithi049.stdout:4416 write 0xba86b8c thru 0xba8bdd8 (0x524d bytes) 2024-04-24T18:08:45.861 INFO:teuthology.orchestra.run.smithi049.stdout:4417 read 0x5999709 thru 0x59a4cc4 (0xb5bc bytes) 2024-04-24T18:08:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:4418 read 0x59b54e3 thru 0x59ba962 (0x5480 bytes) 2024-04-24T18:08:45.862 INFO:teuthology.orchestra.run.smithi049.stdout:4421 writesame 0x3faa7f2 thru 0x3faece7 data_size 0x1a(0x44f6 bytes) 2024-04-24T18:08:45.865 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:45.872+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:45.866 INFO:teuthology.orchestra.run.smithi049.stdout:4422 read 0x670f4a7 thru 0x67168ac (0x7406 bytes) 2024-04-24T18:08:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:4423 read 0x4a13623 thru 0x4a1ec83 (0xb661 bytes) 2024-04-24T18:08:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:4426 punch from 0x7cc9207 to 0x7ccd11c, (0x3f15 bytes) 2024-04-24T18:08:45.867 INFO:teuthology.orchestra.run.smithi049.stdout:4428 writesame 0xb5bcde7 thru 0xb5c9962 data_size 0x12(0xcb7c bytes) 2024-04-24T18:08:45.871 INFO:teuthology.orchestra.run.smithi049.stdout:4430 punch from 0xab7da81 to 0xab7f8f9, (0x1e78 bytes) 2024-04-24T18:08:45.879 INFO:teuthology.orchestra.run.smithi049.stdout:4431 trunc from 0xd216243 to 0x695d574 2024-04-24T18:08:45.890 INFO:teuthology.orchestra.run.smithi049.stdout:4432 punch from 0x1e6d839 to 0x1e7a1ef, (0xc9b6 bytes) 2024-04-24T18:08:45.890 INFO:teuthology.orchestra.run.smithi049.stdout:4433 read 0x94c0ec thru 0x95206e (0x5f83 bytes) 2024-04-24T18:08:45.892 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:45.901 INFO:teuthology.orchestra.run.smithi049.stdout:4434 write 0xa2abae5 thru 0xa2b89aa (0xcec6 bytes) 2024-04-24T18:08:45.904 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:45.904 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:45.904 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:45.904 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:45.904 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:45.905 INFO:teuthology.orchestra.run.smithi049.stdout:4435 writesame 0x528e55d thru 0x52937c8 data_size 0x19(0x526c bytes) 2024-04-24T18:08:45.915 INFO:teuthology.orchestra.run.smithi049.stdout:4438 trunc from 0xa2b89ab to 0x5bb939f 2024-04-24T18:08:45.921 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:45.926 INFO:teuthology.orchestra.run.smithi049.stdout:4439 write 0x547a8b3 thru 0x54814e3 (0x6c31 bytes) 2024-04-24T18:08:45.935 INFO:teuthology.orchestra.run.smithi049.stdout:4440 trunc from 0x5bb939f to 0x88bb46d 2024-04-24T18:08:45.953 INFO:teuthology.orchestra.run.smithi049.stdout:4443 write 0xdcb2e94 thru 0xdcbad8f (0x7efc bytes) 2024-04-24T18:08:45.957 INFO:teuthology.orchestra.run.smithi049.stdout:4444 write 0xd386c1b thru 0xd392cd5 (0xc0bb bytes) 2024-04-24T18:08:45.965 INFO:teuthology.orchestra.run.smithi049.stdout:4447 trunc from 0xdcbad90 to 0xa9ee6e1 2024-04-24T18:08:45.966 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:45.973+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:45.976 INFO:teuthology.orchestra.run.smithi049.stdout:4449 punch from 0x2c8e298 to 0x2c8f4cb, (0x1233 bytes) 2024-04-24T18:08:45.976 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:45.984 INFO:teuthology.orchestra.run.smithi049.stdout:4453 write 0xca7f0ee thru 0xca872eb (0x81fe bytes) 2024-04-24T18:08:45.998 INFO:teuthology.orchestra.run.smithi049.stdout:4454 trunc from 0xca872ec to 0x3bc3cd3 2024-04-24T18:08:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:4459 punch from 0x2543adc to 0x254bc3b, (0x815f bytes) 2024-04-24T18:08:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.011 INFO:teuthology.orchestra.run.smithi049.stdout:4462 writesame 0x111cb84 thru 0x112a5e4 data_size 0x5(0xda61 bytes) 2024-04-24T18:08:46.017 INFO:teuthology.orchestra.run.smithi049.stdout:4463 read 0x17913ce thru 0x179414e (0x2d81 bytes) 2024-04-24T18:08:46.017 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.024 INFO:teuthology.orchestra.run.smithi049.stdout:4465 write 0x484a2d1 thru 0x4852890 (0x85c0 bytes) 2024-04-24T18:08:46.029 INFO:teuthology.orchestra.run.smithi049.stdout:4466 punch from 0x22eb160 to 0x22ec57e, (0x141e bytes) 2024-04-24T18:08:46.038 INFO:teuthology.orchestra.run.smithi049.stdout:4467 writesame 0x698e36a thru 0x699e2c7 data_size 0x1(0xff5e bytes) 2024-04-24T18:08:46.038 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.046 INFO:teuthology.orchestra.run.smithi049.stdout:4471 writesame 0x7c2a254 thru 0x7c39d52 data_size 0x5(0xfaff bytes) 2024-04-24T18:08:46.057 INFO:teuthology.orchestra.run.smithi049.stdout:4473 write 0xb8f14fb thru 0xb8f8672 (0x7178 bytes) 2024-04-24T18:08:46.061 INFO:teuthology.orchestra.run.smithi049.stdout:4474 read 0xae20b5b thru 0xae26fa9 (0x644f bytes) 2024-04-24T18:08:46.066 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:46.074+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:46.068 INFO:teuthology.orchestra.run.smithi049.stdout:4475 write 0xc86d2d6 thru 0xc877d80 (0xaaab bytes) 2024-04-24T18:08:46.072 INFO:teuthology.orchestra.run.smithi049.stdout:4476 write 0x608b89f thru 0x6098227 (0xc989 bytes) 2024-04-24T18:08:46.076 INFO:teuthology.orchestra.run.smithi049.stdout:4478 punch from 0xbc95c01 to 0xbca1323, (0xb722 bytes) 2024-04-24T18:08:46.077 INFO:teuthology.orchestra.run.smithi049.stdout:4479 read 0x20541cb thru 0x205b7b0 (0x75e6 bytes) 2024-04-24T18:08:46.077 INFO:teuthology.orchestra.run.smithi049.stdout:4482 writesame 0x5568eb thru 0x55b0c8 data_size 0x1(0x47de bytes) 2024-04-24T18:08:46.081 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:46.091 INFO:teuthology.orchestra.run.smithi049.stdout:4483 writesame 0xd962bf7 thru 0xd967d94 data_size 0x2(0x519e bytes) 2024-04-24T18:08:46.091 INFO:teuthology.orchestra.run.smithi049.stdout:4484 write 0x7e2b3fd thru 0x7e358f7 (0xa4fb bytes) 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.923176515000002, 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.903537389, 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.903505666999997, 2024-04-24T18:08:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.903482250000003, 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.903452176999998, 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902407029999999, 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902389798999998, 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902372800999999, 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902354119999998, 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902336652000002, 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902320527999997, 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:46.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:46.139 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902289422000003, 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902270221000002, 2024-04-24T18:08:46.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:46.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.902249179000002, 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:46.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.901494231000001, 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:46.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:46.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:46.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:46.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.901467728999997, 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.155 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:46.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.901450199000003, 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.159 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:46.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:46.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:46.162 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.899987039000003, 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.888148391000001, 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.167 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 33.888132048000003, 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:46.171 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:46.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:46.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:46.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:46.175 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:46.176 INFO:teuthology.orchestra.run.smithi049.stdout:4485 read 0xc2b1098 thru 0xc2b94ca (0x8433 bytes) 2024-04-24T18:08:46.176 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:46.177 INFO:teuthology.orchestra.run.smithi049.stdout:4486 trunc from 0xd967d95 to 0x57e7fc7 2024-04-24T18:08:46.178 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:46.178 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.178 INFO:teuthology.orchestra.run.smithi049.stdout:4488 trunc from 0x57e7fc7 to 0x32228a9 2024-04-24T18:08:46.178 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:46.179 INFO:teuthology.orchestra.run.smithi049.stdout:4489 writesame 0xacc43ed thru 0xacc7677 data_size 0x13(0x328b bytes) 2024-04-24T18:08:46.179 INFO:teuthology.orchestra.run.smithi049.stdout:4490 read 0x8ffbeae thru 0x9000509 (0x465c bytes) 2024-04-24T18:08:46.179 INFO:teuthology.orchestra.run.smithi049.stdout:4491 writesame 0x6f3910c thru 0x6f3a9ef data_size 0x36(0x18e4 bytes) 2024-04-24T18:08:46.179 INFO:teuthology.orchestra.run.smithi049.stdout:4492 read 0x9b1b7de thru 0x9b29661 (0xde84 bytes) 2024-04-24T18:08:46.179 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4494 writesame 0xbebc171 thru 0xbecb821 data_size 0x9(0xf6b1 bytes) 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4495 read 0x4c4f565 thru 0x4c51100 (0x1b9c bytes) 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4497 read 0x4f8378b thru 0x4f90b12 (0xd388 bytes) 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4498 trunc from 0xbecb822 to 0xe0a4ff1 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4501 write 0x4653dae thru 0x4656482 (0x26d5 bytes) 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4503 read 0x52be82d thru 0x52ca2de (0xbab2 bytes) 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4504 write 0xbe06335 thru 0xbe07aee (0x17ba bytes) 2024-04-24T18:08:46.181 INFO:teuthology.orchestra.run.smithi049.stdout:4505 write 0x921be18 thru 0x922aa0b (0xebf4 bytes) 2024-04-24T18:08:46.182 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:46.182 INFO:teuthology.orchestra.run.smithi049.stdout:4507 writesame 0x48237f3 thru 0x4827e06 data_size 0x4e(0x4614 bytes) 2024-04-24T18:08:46.182 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:46.184+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:46.187 INFO:teuthology.orchestra.run.smithi049.stdout:4509 trunc from 0xe0a4ff1 to 0x39a241d 2024-04-24T18:08:46.192 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:46.211 INFO:teuthology.orchestra.run.smithi049.stdout:4510 writesame 0xa8f4759 thru 0xa8f9281 data_size 0x1(0x4b29 bytes) 2024-04-24T18:08:46.211 INFO:teuthology.orchestra.run.smithi049.stdout:4511 read 0x47ebb0c thru 0x47ebf9b (0x490 bytes) 2024-04-24T18:08:46.211 INFO:teuthology.orchestra.run.smithi049.stdout:4512 read 0x86252ad thru 0x86334a7 (0xe1fb bytes) 2024-04-24T18:08:46.211 INFO:teuthology.orchestra.run.smithi049.stdout:4513 punch from 0xa4a6a78 to 0xa4b64fe, (0xfa86 bytes) 2024-04-24T18:08:46.211 INFO:teuthology.orchestra.run.smithi049.stdout:4515 read 0x4dad60 thru 0x4e8e1a (0xe0bb bytes) 2024-04-24T18:08:46.211 INFO:teuthology.orchestra.run.smithi049.stdout:4516 writesame 0x1595092 thru 0x1598a58 data_size 0x1(0x39c7 bytes) 2024-04-24T18:08:46.215 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:46.216 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:46.219 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:46.219 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:46.222 INFO:teuthology.orchestra.run.smithi049.stdout:4517 trunc from 0xa8f9282 to 0xc0b18cc 2024-04-24T18:08:46.227 INFO:teuthology.orchestra.run.smithi049.stdout:4518 read 0xacb6b00 thru 0xacb6bce (0xcf bytes) 2024-04-24T18:08:46.228 INFO:teuthology.orchestra.run.smithi049.stdout:4519 read 0xbdc1f89 thru 0xbdcab53 (0x8bcb bytes) 2024-04-24T18:08:46.228 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.228 INFO:teuthology.orchestra.run.smithi049.stdout:4523 write 0x13817a thru 0x13f55e (0x73e5 bytes) 2024-04-24T18:08:46.260 INFO:teuthology.orchestra.run.smithi049.stdout:4524 writesame 0x589c3b7 thru 0x589f74e data_size 0x1a(0x3398 bytes) 2024-04-24T18:08:46.263 INFO:teuthology.orchestra.run.smithi049.stdout:4525 read 0x450a8fe thru 0x4518a2d (0xe130 bytes) 2024-04-24T18:08:46.263 INFO:teuthology.orchestra.run.smithi049.stdout:4526 read 0xb20a7ea thru 0xb216f45 (0xc75c bytes) 2024-04-24T18:08:46.263 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.263 INFO:teuthology.orchestra.run.smithi049.stdout:4529 punch from 0xa63b036 to 0xa642243, (0x720d bytes) 2024-04-24T18:08:46.268 INFO:teuthology.orchestra.run.smithi049.stdout:4530 trunc from 0xc0b18cc to 0x785ce49 2024-04-24T18:08:46.277 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:46.285+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:46.281 INFO:teuthology.orchestra.run.smithi049.stdout:4533 write 0x97f352a thru 0x9800f46 (0xda1d bytes) 2024-04-24T18:08:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:4534 writesame 0xe9e128a thru 0xe9edec2 data_size 0x6f(0xcc39 bytes) 2024-04-24T18:08:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:4537 read 0x74da579 thru 0x74dafdc (0xa64 bytes) 2024-04-24T18:08:46.297 INFO:teuthology.orchestra.run.smithi049.stdout:4539 punch from 0x6d10552 to 0x6d15d83, (0x5831 bytes) 2024-04-24T18:08:46.298 INFO:teuthology.orchestra.run.smithi049.stdout:4540 write 0x8b9e4ec thru 0x8ba6c54 (0x8769 bytes) 2024-04-24T18:08:46.300 INFO:teuthology.orchestra.run.smithi049.stdout:4541 writesame 0x47ff6e1 thru 0x4800a7d data_size 0x1(0x139d bytes) 2024-04-24T18:08:46.304 INFO:teuthology.orchestra.run.smithi049.stdout:4543 read 0x6fac876 thru 0x6fb5650 (0x8ddb bytes) 2024-04-24T18:08:46.305 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.305 INFO:teuthology.orchestra.run.smithi049.stdout:4547 write 0x19691bf thru 0x196c4d3 (0x3315 bytes) 2024-04-24T18:08:46.308 INFO:teuthology.orchestra.run.smithi049.stdout:4548 read 0xa1032b0 thru 0xa103906 (0x657 bytes) 2024-04-24T18:08:46.308 INFO:teuthology.orchestra.run.smithi049.stdout:4549 read 0x5379e6c thru 0x5382489 (0x861e bytes) 2024-04-24T18:08:46.315 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:46.315 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:46.315 INFO:teuthology.orchestra.run.smithi049.stdout:4550 trunc from 0xe9edec3 to 0x7eed58f 2024-04-24T18:08:46.326 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:46.326 INFO:teuthology.orchestra.run.smithi049.stdout:4551 write 0x27e3403 thru 0x27ef695 (0xc293 bytes) 2024-04-24T18:08:46.338 INFO:teuthology.orchestra.run.smithi049.stdout:4552 writesame 0xa97a2e3 thru 0xa981bf5 data_size 0x1(0x7913 bytes) 2024-04-24T18:08:46.338 INFO:teuthology.orchestra.run.smithi049.stdout:4553 read 0xa4c4d93 thru 0xa4c7b91 (0x2dff bytes) 2024-04-24T18:08:46.338 INFO:teuthology.orchestra.run.smithi049.stdout:4554 write 0xdc55d2 thru 0xdd371f (0xe14e bytes) 2024-04-24T18:08:46.339 INFO:teuthology.orchestra.run.smithi049.stdout:4555 write 0x51865e6 thru 0x5186dfb (0x816 bytes) 2024-04-24T18:08:46.342 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.342 INFO:teuthology.orchestra.run.smithi049.stdout:4557 read 0x3ee327c thru 0x3eed848 (0xa5cd bytes) 2024-04-24T18:08:46.342 INFO:teuthology.orchestra.run.smithi049.stdout:4558 write 0x304494c thru 0x304f87c (0xaf31 bytes) 2024-04-24T18:08:46.346 INFO:teuthology.orchestra.run.smithi049.stdout:4559 read 0x8aaf278 thru 0x8ab1fbd (0x2d46 bytes) 2024-04-24T18:08:46.346 INFO:teuthology.orchestra.run.smithi049.stdout:4560 read 0x9ee76a4 thru 0x9eedaf3 (0x6450 bytes) 2024-04-24T18:08:46.346 INFO:teuthology.orchestra.run.smithi049.stdout:4561 write 0x2a090e7 thru 0x2a17d4e (0xec68 bytes) 2024-04-24T18:08:46.349 INFO:teuthology.orchestra.run.smithi049.stdout:4562 read 0x1e50c3c thru 0x1e603fd (0xf7c2 bytes) 2024-04-24T18:08:46.350 INFO:teuthology.orchestra.run.smithi049.stdout:4563 punch from 0x73a91e0 to 0x73b1f2e, (0x8d4e bytes) 2024-04-24T18:08:46.350 INFO:teuthology.orchestra.run.smithi049.stdout:4564 write 0x52f1cec thru 0x52fe72a (0xca3f bytes) 2024-04-24T18:08:46.361 INFO:teuthology.orchestra.run.smithi049.stdout:4566 writesame 0xd03c5e8 thru 0xd04988e data_size 0x1(0xd2a7 bytes) 2024-04-24T18:08:46.361 INFO:teuthology.orchestra.run.smithi049.stdout:4567 read 0x91fb817 thru 0x91ffe5f (0x4649 bytes) 2024-04-24T18:08:46.361 INFO:teuthology.orchestra.run.smithi049.stdout:4568 write 0xa065a94 thru 0xa0666e4 (0xc51 bytes) 2024-04-24T18:08:46.363 INFO:teuthology.orchestra.run.smithi049.stdout:4569 writesame 0x805ecb thru 0x813120 data_size 0x6d(0xd256 bytes) 2024-04-24T18:08:46.379 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:46.386+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:46.411 INFO:teuthology.orchestra.run.smithi049.stdout:4570 writesame 0x53c5412 thru 0x53d3af5 data_size 0xe(0xe6e4 bytes) 2024-04-24T18:08:46.413 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.415 INFO:teuthology.orchestra.run.smithi049.stdout:4572 trunc from 0xd04988f to 0xed08816 2024-04-24T18:08:46.421 INFO:teuthology.orchestra.run.smithi049.stdout:4573 write 0x755cb5d thru 0x7568e81 (0xc325 bytes) 2024-04-24T18:08:46.425 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.425 INFO:teuthology.orchestra.run.smithi049.stdout:4576 writesame 0x7a766ac thru 0x7a819f0 data_size 0x1(0xb345 bytes) 2024-04-24T18:08:46.439 INFO:teuthology.orchestra.run.smithi049.stdout:4577 read 0x4bd4ab7 thru 0x4bda6e3 (0x5c2d bytes) 2024-04-24T18:08:46.440 INFO:teuthology.orchestra.run.smithi049.stdout:4579 writesame 0x9d165b thru 0x9d5853 data_size 0x1(0x41f9 bytes) 2024-04-24T18:08:46.447 INFO:teuthology.orchestra.run.smithi049.stdout:4580 read 0x77a46ed thru 0x77b2b98 (0xe4ac bytes) 2024-04-24T18:08:46.461 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:46.461 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:46.462 INFO:teuthology.orchestra.run.smithi049.stdout:4581 trunc from 0xed08816 to 0x1e47215 2024-04-24T18:08:46.473 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:46.475 INFO:teuthology.orchestra.run.smithi049.stdout:4582 read 0xb6e4a7 thru 0xb795ac (0xb106 bytes) 2024-04-24T18:08:46.476 INFO:teuthology.orchestra.run.smithi049.stdout:4583 punch from 0x32cf22 to 0x33c5a7, (0xf685 bytes) 2024-04-24T18:08:46.476 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.481 INFO:teuthology.orchestra.run.smithi049.stdout:4585 trunc from 0x1e47215 to 0x6b02b4f 2024-04-24T18:08:46.487 INFO:teuthology.orchestra.run.smithi049.stdout:4589 trunc from 0x6b02b4f to 0x5c2e4cf 2024-04-24T18:08:46.503 INFO:teuthology.orchestra.run.smithi049.stdout:4590 writesame 0xa938588 thru 0xa93cbf6 data_size 0x13(0x466f bytes) 2024-04-24T18:08:46.513 INFO:teuthology.orchestra.run.smithi049.stdout:4591 writesame 0xed048dc thru 0xed0fa92 data_size 0x2d(0xb1b7 bytes) 2024-04-24T18:08:46.516 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:46.525+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:46.528 INFO:teuthology.orchestra.run.smithi049.stdout:4592 trunc from 0xed0fa93 to 0x1dfc925 2024-04-24T18:08:46.534 INFO:teuthology.orchestra.run.smithi049.stdout:4593 punch from 0x3365fd to 0x341074, (0xaa77 bytes) 2024-04-24T18:08:46.534 INFO:teuthology.orchestra.run.smithi049.stdout:4594 punch from 0x10c175b to 0x10c6205, (0x4aaa bytes) 2024-04-24T18:08:46.534 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.545 INFO:teuthology.orchestra.run.smithi049.stdout:4596 trunc from 0x1dfc925 to 0xc2230b4 2024-04-24T18:08:46.560 INFO:teuthology.orchestra.run.smithi049.stdout:4597 trunc from 0xc2230b4 to 0x43008ab 2024-04-24T18:08:46.565 INFO:teuthology.orchestra.run.smithi049.stdout:4600 write 0x2c43547 thru 0x2c47034 (0x3aee bytes) 2024-04-24T18:08:46.574 INFO:teuthology.orchestra.run.smithi049.stdout:4601 trunc from 0x43008ab to 0x8d98c9f 2024-04-24T18:08:46.579 INFO:teuthology.orchestra.run.smithi049.stdout:4603 read 0x6bef30f thru 0x6bf23c4 (0x30b6 bytes) 2024-04-24T18:08:46.579 INFO:teuthology.orchestra.run.smithi049.stdout:4604 read 0x6e354b2 thru 0x6e391f2 (0x3d41 bytes) 2024-04-24T18:08:46.580 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:46.588+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:46.587 INFO:teuthology.orchestra.run.smithi049.stdout:4605 write 0xb4023c3 thru 0xb40ee2b (0xca69 bytes) 2024-04-24T18:08:46.591 INFO:teuthology.orchestra.run.smithi049.stdout:4608 read 0x6abd56c thru 0x6ac90e2 (0xbb77 bytes) 2024-04-24T18:08:46.591 INFO:teuthology.orchestra.run.smithi049.stdout:4609 write 0x6b0f960 thru 0x6b0fff0 (0x691 bytes) 2024-04-24T18:08:46.593 INFO:teuthology.orchestra.run.smithi049.stdout:4612 write 0xb19e857 thru 0xb1a029b (0x1a45 bytes) 2024-04-24T18:08:46.596 INFO:teuthology.orchestra.run.smithi049.stdout:4613 write 0x1f0b7a2 thru 0x1f1a52a (0xed89 bytes) 2024-04-24T18:08:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:4614 punch from 0x42793c8 to 0x427be8a, (0x2ac2 bytes) 2024-04-24T18:08:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:4615 punch from 0x55c6047 to 0x55d17a4, (0xb75d bytes) 2024-04-24T18:08:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:4619 read 0x5af00be thru 0x5afaf03 (0xae46 bytes) 2024-04-24T18:08:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:4620 read 0x727ad4f thru 0x7285ea5 (0xb157 bytes) 2024-04-24T18:08:46.598 INFO:teuthology.orchestra.run.smithi049.stdout:4622 write 0x2b16bd4 thru 0x2b21284 (0xa6b1 bytes) 2024-04-24T18:08:46.599 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:46.599 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:46.601 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.601 INFO:teuthology.orchestra.run.smithi049.stdout:4628 writesame 0x1a27bde thru 0x1a2978c data_size 0x1(0x1baf bytes) 2024-04-24T18:08:46.604 INFO:teuthology.orchestra.run.smithi049.stdout:4629 write 0x49d0358 thru 0x49db3e6 (0xb08f bytes) 2024-04-24T18:08:46.608 INFO:teuthology.orchestra.run.smithi049.stdout:4630 read 0x4c15f6c thru 0x4c16f78 (0x100d bytes) 2024-04-24T18:08:46.608 INFO:teuthology.orchestra.run.smithi049.stdout:4631 punch from 0x2a5cd2b to 0x2a691c7, (0xc49c bytes) 2024-04-24T18:08:46.608 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.608 INFO:teuthology.orchestra.run.smithi049.stdout:4634 write 0x2dc9f1d thru 0x2dcc7a7 (0x288b bytes) 2024-04-24T18:08:46.609 INFO:teuthology.orchestra.run.smithi049.stdout:4636 writesame 0x473d182 thru 0x4745ac3 data_size 0x1(0x8942 bytes) 2024-04-24T18:08:46.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:46.624 INFO:teuthology.orchestra.run.smithi049.stdout:4637 read 0x35a1d23 thru 0x35ad253 (0xb531 bytes) 2024-04-24T18:08:46.635 INFO:teuthology.orchestra.run.smithi049.stdout:4638 trunc from 0xb40ee2c to 0x16532f4 2024-04-24T18:08:46.662 INFO:teuthology.orchestra.run.smithi049.stdout:4641 write 0x8ccabcd thru 0x8cd84fe (0xd932 bytes) 2024-04-24T18:08:46.666 INFO:teuthology.orchestra.run.smithi049.stdout:4642 trunc from 0x8cd84ff to 0x9a7dc7a 2024-04-24T18:08:46.678 INFO:teuthology.orchestra.run.smithi049.stdout:4643 writesame 0xb73453f thru 0xb73a508 data_size 0x3(0x5fca bytes) 2024-04-24T18:08:46.679 INFO:teuthology.orchestra.run.smithi049.stdout:4644 read 0x1e70e75 thru 0x1e7f92e (0xeaba bytes) 2024-04-24T18:08:46.679 INFO:teuthology.orchestra.run.smithi049.stdout:4645 read 0x98483a7 thru 0x9855373 (0xcfcd bytes) 2024-04-24T18:08:46.688 INFO:teuthology.orchestra.run.smithi049.stdout:4646 write 0xedf394f thru 0xedf4308 (0x9ba bytes) 2024-04-24T18:08:46.692 INFO:teuthology.orchestra.run.smithi049.stdout:4647 read 0xdbf9bb2 thru 0xdc05c02 (0xc051 bytes) 2024-04-24T18:08:46.692 INFO:teuthology.orchestra.run.smithi049.stdout:4648 punch from 0x89aa2c5 to 0x89b0b25, (0x6860 bytes) 2024-04-24T18:08:46.692 INFO:teuthology.orchestra.run.smithi049.stdout:4649 read 0x5540eaa thru 0x5547396 (0x64ed bytes) 2024-04-24T18:08:46.692 INFO:teuthology.orchestra.run.smithi049.stdout:4650 write 0x8ce5ef2 thru 0x8cf088e (0xa99d bytes) 2024-04-24T18:08:46.693 INFO:teuthology.orchestra.run.smithi049.stdout:4651 punch from 0x3f68a4c to 0x3f75ad7, (0xd08b bytes) 2024-04-24T18:08:46.694 INFO:teuthology.orchestra.run.smithi049.stdout:4652 write 0x3f402b8 thru 0x3f4a3f2 (0xa13b bytes) 2024-04-24T18:08:46.694 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:46.703+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:46.697 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.697 INFO:teuthology.orchestra.run.smithi049.stdout:4654 read 0x131d6a3 thru 0x1328514 (0xae72 bytes) 2024-04-24T18:08:46.702 INFO:teuthology.orchestra.run.smithi049.stdout:4655 trunc from 0xedf4309 to 0xab4ae5e 2024-04-24T18:08:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:4658 read 0x4c81a21 thru 0x4c85831 (0x3e11 bytes) 2024-04-24T18:08:46.712 INFO:teuthology.orchestra.run.smithi049.stdout:4660 write 0x65ddb03 thru 0x65decaa (0x11a8 bytes) 2024-04-24T18:08:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:4661 punch from 0x500c9c1 to 0x5010c86, (0x42c5 bytes) 2024-04-24T18:08:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:4662 read 0x4186cde thru 0x4194b88 (0xdeab bytes) 2024-04-24T18:08:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:4664 read 0x19dcf0e thru 0x19dd524 (0x617 bytes) 2024-04-24T18:08:46.715 INFO:teuthology.orchestra.run.smithi049.stdout:4665 writesame 0x2ed76f3 thru 0x2ee33ee data_size 0x52(0xbcfc bytes) 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:4667 punch from 0x3695bdc to 0x36a2881, (0xcca5 bytes) 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:4671 read 0x1e4e849 thru 0x1e52183 (0x393b bytes) 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:4672 read 0x1947803 thru 0x1953859 (0xc057 bytes) 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:4673 read 0x2298089 thru 0x22a4f3d (0xceb5 bytes) 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:4675 read 0x5baf7f4 thru 0x5bb2405 (0x2c12 bytes) 2024-04-24T18:08:46.720 INFO:teuthology.orchestra.run.smithi049.stdout:4676 write 0x8a78634 thru 0x8a8817f (0xfb4c bytes) 2024-04-24T18:08:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.723 INFO:teuthology.orchestra.run.smithi049.stdout:4680 writesame 0x12ba08b thru 0x12c929e data_size 0x2(0xf214 bytes) 2024-04-24T18:08:46.734 INFO:teuthology.orchestra.run.smithi049.stdout:4681 write 0x99a4fdc thru 0x99a7f66 (0x2f8b bytes) 2024-04-24T18:08:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:4683 read 0x771916f thru 0x7719b5a (0x9ec bytes) 2024-04-24T18:08:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:4685 punch from 0x702aaff to 0x7031605, (0x6b06 bytes) 2024-04-24T18:08:46.737 INFO:teuthology.orchestra.run.smithi049.stdout:4687 read 0x3c0515 thru 0x3c5cef (0x57db bytes) 2024-04-24T18:08:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:4688 read 0x4240f08 thru 0x424fb4b (0xec44 bytes) 2024-04-24T18:08:46.738 INFO:teuthology.orchestra.run.smithi049.stdout:4689 trunc from 0xab4ae5e to 0xb6a6789 2024-04-24T18:08:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:4690 read 0xa022b34 thru 0xa0324a8 (0xf975 bytes) 2024-04-24T18:08:46.744 INFO:teuthology.orchestra.run.smithi049.stdout:4691 writesame 0x66884a8 thru 0x668b84e data_size 0x1(0x33a7 bytes) 2024-04-24T18:08:46.752 INFO:teuthology.orchestra.run.smithi049.stdout:4693 punch from 0xaaa0ad2 to 0xaaa2a01, (0x1f2f bytes) 2024-04-24T18:08:46.753 INFO:teuthology.orchestra.run.smithi049.stdout:4695 trunc from 0xb6a6789 to 0xa5f6a18 2024-04-24T18:08:46.758 INFO:teuthology.orchestra.run.smithi049.stdout:4696 read 0x8d9ca65 thru 0x8da8bdc (0xc178 bytes) 2024-04-24T18:08:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:4697 writesame 0xcfdab9e thru 0xcfea61b data_size 0x1(0xfa7e bytes) 2024-04-24T18:08:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:4698 read 0xad86201 thru 0xad93063 (0xce63 bytes) 2024-04-24T18:08:46.767 INFO:teuthology.orchestra.run.smithi049.stdout:4699 trunc from 0xcfea61c to 0xd981ebb 2024-04-24T18:08:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:4700 punch from 0x60524bb to 0x6054ecf, (0x2a14 bytes) 2024-04-24T18:08:46.773 INFO:teuthology.orchestra.run.smithi049.stdout:4701 read 0x2af716 thru 0x2b4937 (0x5222 bytes) 2024-04-24T18:08:46.775 INFO:teuthology.orchestra.run.smithi049.stdout:4702 trunc from 0xd981ebb to 0xc0bf915 2024-04-24T18:08:46.778 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:46.778 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:46.781 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:46.790+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:4704 writesame 0xc140f06 thru 0xc143bc6 data_size 0x9(0x2cc1 bytes) 2024-04-24T18:08:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:4706 read 0xbd12429 thru 0xbd1cc0f (0xa7e7 bytes) 2024-04-24T18:08:46.786 INFO:teuthology.orchestra.run.smithi049.stdout:4707 write 0x29d48a1 thru 0x29e454a (0xfcaa bytes) 2024-04-24T18:08:46.789 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:46.791 INFO:teuthology.orchestra.run.smithi049.stdout:4708 punch from 0x52950f to 0x532fbe, (0x9aaf bytes) 2024-04-24T18:08:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:4710 punch from 0xa53385d to 0xa542b0b, (0xf2ae bytes) 2024-04-24T18:08:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:4711 read 0x3f63e7e thru 0x3f6e9cd (0xab50 bytes) 2024-04-24T18:08:46.792 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:4713 punch from 0xa054fb8 to 0xa05ab92, (0x5bda bytes) 2024-04-24T18:08:46.793 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:4718 read 0x912df2d thru 0x913d9f7 (0xfacb bytes) 2024-04-24T18:08:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:4722 read 0xacda8b1 thru 0xacde5f1 (0x3d41 bytes) 2024-04-24T18:08:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:4725 write 0x1ebdd43 thru 0x1eca9df (0xcc9d bytes) 2024-04-24T18:08:46.794 INFO:teuthology.orchestra.run.smithi049.stdout:4726 clone 44 order 25 su 131072 sc 15 2024-04-24T18:08:46.880 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [6] live_osds: [0, 1, 2, 3, 4, 5, 7] 2024-04-24T18:08:46.881 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-04-24T18:08:46.881 INFO:tasks.ceph.osd.6:Restarting daemon 2024-04-24T18:08:46.881 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-04-24T18:08:46.882 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:46.890+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:46.883 INFO:tasks.ceph.osd.6:Started 2024-04-24T18:08:46.883 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:46.893 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone43 from 0xc143bc7 (overlap 0x1266cda) to 0x5853db 2024-04-24T18:08:46.960 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #42, image image_client.0-clone42 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent43 2024-04-24T18:08:46.965 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:46.965 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:46.976 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:46.983 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:46.991+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:47.010 INFO:teuthology.orchestra.run.smithi078.stderr:no valid command found; 10 closest matches: 2024-04-24T18:08:47.010 INFO:teuthology.orchestra.run.smithi078.stderr:0 2024-04-24T18:08:47.010 INFO:teuthology.orchestra.run.smithi078.stderr:1 2024-04-24T18:08:47.010 INFO:teuthology.orchestra.run.smithi078.stderr:2 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:abort 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:assert 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:config diff 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:config diff get 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:config get 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:config help [] 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:config set ... 2024-04-24T18:08:47.011 INFO:teuthology.orchestra.run.smithi078.stderr:admin_socket: invalid command 2024-04-24T18:08:47.012 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-24T18:08:47.012 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-04-24T18:08:47.084 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:47.092+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:47.105 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:47.106 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:47.115 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:47.185 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:47.193+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:47.240 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:47.240 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:47.249 INFO:teuthology.orchestra.run.smithi049.stdout:4730 writesame 0x956fd0 thru 0x95a886 data_size 0x1(0x38b7 bytes) 2024-04-24T18:08:47.250 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:47.269 INFO:teuthology.orchestra.run.smithi049.stdout:4732 writesame 0x39fb75a thru 0x3a0751e data_size 0x25(0xbdc5 bytes) 2024-04-24T18:08:47.276 INFO:teuthology.orchestra.run.smithi049.stdout:4734 writesame 0x5c61e thru 0x65eef data_size 0x1f(0x98d2 bytes) 2024-04-24T18:08:47.285 INFO:teuthology.orchestra.run.smithi049.stdout:4735 read 0xbe298c6 thru 0xbe3792d (0xe068 bytes) 2024-04-24T18:08:47.285 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.285 INFO:teuthology.orchestra.run.smithi049.stdout:4738 read 0x4f322cd thru 0x4f3c3f4 (0xa128 bytes) 2024-04-24T18:08:47.285 INFO:teuthology.orchestra.run.smithi049.stdout:4740 write 0x1ec9602 thru 0x1ed28b8 (0x92b7 bytes) 2024-04-24T18:08:47.286 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:47.293+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:47.293 INFO:teuthology.orchestra.run.smithi049.stdout:4741 write 0xa704f67 thru 0xa70bf7b (0x7015 bytes) 2024-04-24T18:08:47.308 INFO:teuthology.orchestra.run.smithi049.stdout:4742 write 0xebae6b4 thru 0xebb4845 (0x6192 bytes) 2024-04-24T18:08:47.314 INFO:teuthology.orchestra.run.smithi049.stdout:4743 writesame 0x6c838bc thru 0x6c859ab data_size 0x8(0x20f0 bytes) 2024-04-24T18:08:47.315 INFO:teuthology.orchestra.run.smithi049.stdout:4746 write 0x9a47c4f thru 0x9a49597 (0x1949 bytes) 2024-04-24T18:08:47.322 INFO:teuthology.orchestra.run.smithi049.stdout:4747 write 0x852b306 thru 0x8539ebe (0xebb9 bytes) 2024-04-24T18:08:47.332 INFO:teuthology.orchestra.run.smithi049.stdout:4748 trunc from 0xebb4846 to 0xbafd7bb 2024-04-24T18:08:47.363 INFO:teuthology.orchestra.run.smithi049.stdout:4749 read 0x576efe0 thru 0x576f633 (0x654 bytes) 2024-04-24T18:08:47.364 INFO:teuthology.orchestra.run.smithi049.stdout:4750 punch from 0x101283a to 0x101ecd0, (0xc496 bytes) 2024-04-24T18:08:47.364 INFO:teuthology.orchestra.run.smithi049.stdout:4751 punch from 0x6e73978 to 0x6e782b4, (0x493c bytes) 2024-04-24T18:08:47.366 INFO:teuthology.orchestra.run.smithi049.stdout:4752 trunc from 0xbafd7bb to 0xdde641f 2024-04-24T18:08:47.371 INFO:teuthology.orchestra.run.smithi049.stdout:4753 punch from 0xa5ca249 to 0xa5cf183, (0x4f3a bytes) 2024-04-24T18:08:47.371 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.371 INFO:teuthology.orchestra.run.smithi049.stdout:4756 read 0x46505a8 thru 0x4660008 (0xfa61 bytes) 2024-04-24T18:08:47.372 INFO:teuthology.orchestra.run.smithi049.stdout:4760 punch from 0x63b5fae to 0x63bc8b6, (0x6908 bytes) 2024-04-24T18:08:47.378 INFO:teuthology.orchestra.run.smithi049.stdout:4762 write 0xeb6ed9a thru 0xeb7cbbd (0xde24 bytes) 2024-04-24T18:08:47.379 INFO:teuthology.orchestra.run.smithi049.stdout:4764 writesame 0x2c5d225 thru 0x2c62bc7 data_size 0x1(0x59a3 bytes) 2024-04-24T18:08:47.386 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:47.394+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:47.396 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:47.396 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:47.404 INFO:teuthology.orchestra.run.smithi049.stdout:4765 trunc from 0xeb7cbbe to 0x12b6926 2024-04-24T18:08:47.407 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:47.427 INFO:teuthology.orchestra.run.smithi049.stdout:4766 writesame 0x9dbc507 thru 0x9dc3f3c data_size 0x1(0x7a36 bytes) 2024-04-24T18:08:47.435 INFO:teuthology.orchestra.run.smithi049.stdout:4767 trunc from 0x9dc3f3d to 0x3f57c1d 2024-04-24T18:08:47.451 INFO:teuthology.orchestra.run.smithi049.stdout:4768 trunc from 0x3f57c1d to 0xb34b65c 2024-04-24T18:08:47.463 INFO:teuthology.orchestra.run.smithi049.stdout:4769 write 0xcccc404 thru 0xccd7304 (0xaf01 bytes) 2024-04-24T18:08:47.465 INFO:teuthology.orchestra.run.smithi049.stdout:4770 punch from 0x2bf478b to 0x2c01366, (0xcbdb bytes) 2024-04-24T18:08:47.465 INFO:teuthology.orchestra.run.smithi049.stdout:4771 read 0xa289af8 thru 0xa28f195 (0x569e bytes) 2024-04-24T18:08:47.465 INFO:teuthology.orchestra.run.smithi049.stdout:4772 read 0x81da810 thru 0x81e350c (0x8cfd bytes) 2024-04-24T18:08:47.465 INFO:teuthology.orchestra.run.smithi049.stdout:4773 write 0xbefe27b thru 0xbf0e24a (0xffd0 bytes) 2024-04-24T18:08:47.467 INFO:teuthology.orchestra.run.smithi049.stdout:4774 punch from 0x7779be2 to 0x7783709, (0x9b27 bytes) 2024-04-24T18:08:47.467 INFO:teuthology.orchestra.run.smithi049.stdout:4775 punch from 0x20378a8 to 0x203fa86, (0x81de bytes) 2024-04-24T18:08:47.467 INFO:teuthology.orchestra.run.smithi049.stdout:4778 read 0x8306e91 thru 0x8311a5e (0xabce bytes) 2024-04-24T18:08:47.468 INFO:teuthology.orchestra.run.smithi049.stdout:4779 write 0x93b4cc7 thru 0x93be5c1 (0x98fb bytes) 2024-04-24T18:08:47.469 INFO:teuthology.orchestra.run.smithi049.stdout:4780 punch from 0x1bd40fb to 0x1bd6b2d, (0x2a32 bytes) 2024-04-24T18:08:47.482 INFO:teuthology.orchestra.run.smithi049.stdout:4782 trunc from 0xccd7305 to 0x2912619 2024-04-24T18:08:47.485 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:47.493+0000 7f6edd7a9540 -1 Falling back to public interface 2024-04-24T18:08:47.487 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:47.495+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:47.489 INFO:teuthology.orchestra.run.smithi049.stdout:4783 read 0x901bf2 thru 0x907b48 (0x5f57 bytes) 2024-04-24T18:08:47.489 INFO:teuthology.orchestra.run.smithi049.stdout:4784 punch from 0x15f5c7a to 0x15faf05, (0x528b bytes) 2024-04-24T18:08:47.503 INFO:teuthology.orchestra.run.smithi049.stdout:4785 write 0xb1f80e0 thru 0xb1fddef (0x5d10 bytes) 2024-04-24T18:08:47.505 INFO:teuthology.orchestra.run.smithi049.stdout:4786 write 0xa7118e6 thru 0xa71255a (0xc75 bytes) 2024-04-24T18:08:47.506 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.506 INFO:teuthology.orchestra.run.smithi049.stdout:4789 read 0x24b81f5 thru 0x24bba3b (0x3847 bytes) 2024-04-24T18:08:47.507 INFO:teuthology.orchestra.run.smithi049.stdout:4791 write 0x19f04a6 thru 0x19fbd04 (0xb85f bytes) 2024-04-24T18:08:47.508 INFO:teuthology.orchestra.run.smithi049.stdout:4792 write 0x2c42673 thru 0x2c4b194 (0x8b22 bytes) 2024-04-24T18:08:47.510 INFO:teuthology.orchestra.run.smithi049.stdout:4793 punch from 0x692cb69 to 0x6931709, (0x4ba0 bytes) 2024-04-24T18:08:47.510 INFO:teuthology.orchestra.run.smithi049.stdout:4794 read 0xadfa53 thru 0xaef862 (0xfe10 bytes) 2024-04-24T18:08:47.511 INFO:teuthology.orchestra.run.smithi049.stdout:4795 write 0x8bbf9bd thru 0x8bc4c77 (0x52bb bytes) 2024-04-24T18:08:47.521 INFO:teuthology.orchestra.run.smithi049.stdout:4797 write 0xe9b5d39 thru 0xe9c3ae4 (0xddac bytes) 2024-04-24T18:08:47.523 INFO:teuthology.orchestra.run.smithi049.stdout:4798 read 0x296dbf5 thru 0x29734f9 (0x5905 bytes) 2024-04-24T18:08:47.525 INFO:teuthology.orchestra.run.smithi049.stdout:4799 trunc from 0xe9c3ae5 to 0xc8cb00b 2024-04-24T18:08:47.540 INFO:teuthology.orchestra.run.smithi049.stdout:4802 trunc from 0xc8cb00b to 0x69703bc 2024-04-24T18:08:47.540 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:47.540 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:47.547 INFO:teuthology.orchestra.run.smithi049.stdout:4803 writesame 0x333fe4e thru 0x334518d data_size 0x60(0x5340 bytes) 2024-04-24T18:08:47.551 INFO:teuthology.orchestra.run.smithi049.stdout:4804 trunc from 0x69703bc to 0x856b5e5 2024-04-24T18:08:47.552 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:47.568 INFO:teuthology.orchestra.run.smithi049.stdout:4805 writesame 0xdc7e67a thru 0xdc86793 data_size 0x19(0x811a bytes) 2024-04-24T18:08:47.569 INFO:teuthology.orchestra.run.smithi049.stdout:4806 read 0x276f3a5 thru 0x2774ffc (0x5c58 bytes) 2024-04-24T18:08:47.570 INFO:teuthology.orchestra.run.smithi049.stdout:4807 read 0x76c69c thru 0x777663 (0xafc8 bytes) 2024-04-24T18:08:47.570 INFO:teuthology.orchestra.run.smithi049.stdout:4809 write 0x12b56e2 thru 0x12bb398 (0x5cb7 bytes) 2024-04-24T18:08:47.572 INFO:teuthology.orchestra.run.smithi049.stdout:4810 punch from 0xde119b to 0xdeb152, (0x9fb7 bytes) 2024-04-24T18:08:47.572 INFO:teuthology.orchestra.run.smithi049.stdout:4811 write 0x1251018 thru 0x125267f (0x1668 bytes) 2024-04-24T18:08:47.573 INFO:teuthology.orchestra.run.smithi049.stdout:4812 punch from 0xc55e44e to 0xc568afa, (0xa6ac bytes) 2024-04-24T18:08:47.573 INFO:teuthology.orchestra.run.smithi049.stdout:4813 write 0xc2b853c thru 0xc2b92d2 (0xd97 bytes) 2024-04-24T18:08:47.574 INFO:teuthology.orchestra.run.smithi049.stdout:4814 write 0x4c67ed4 thru 0x4c6e0c7 (0x61f4 bytes) 2024-04-24T18:08:47.576 INFO:teuthology.orchestra.run.smithi049.stdout:4815 punch from 0x5aa5f5d to 0x5ab0349, (0xa3ec bytes) 2024-04-24T18:08:47.583 INFO:teuthology.orchestra.run.smithi049.stdout:4817 write 0xebd6e50 thru 0xebdeff9 (0x81aa bytes) 2024-04-24T18:08:47.585 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.585 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.585 INFO:teuthology.orchestra.run.smithi049.stdout:4823 write 0x796e7e thru 0x797074 (0x1f7 bytes) 2024-04-24T18:08:47.586 INFO:teuthology.orchestra.run.smithi049.stdout:4824 read 0xdb00e14 thru 0xdb05473 (0x4660 bytes) 2024-04-24T18:08:47.586 INFO:teuthology.orchestra.run.smithi049.stdout:4826 punch from 0x63bf7d0 to 0x63c68c3, (0x70f3 bytes) 2024-04-24T18:08:47.586 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.587 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.593 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:47.601+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:47.597 INFO:teuthology.orchestra.run.smithi049.stdout:4832 trunc from 0xebdeffa to 0x51e3750 2024-04-24T18:08:47.605 INFO:teuthology.orchestra.run.smithi049.stdout:4833 read 0x2e7b0fa thru 0x2e7d89f (0x27a6 bytes) 2024-04-24T18:08:47.606 INFO:teuthology.orchestra.run.smithi049.stdout:4836 punch from 0x495fdf1 to 0x496791d, (0x7b2c bytes) 2024-04-24T18:08:47.606 INFO:teuthology.orchestra.run.smithi049.stdout:4838 read 0x41a0fad thru 0x41a1bd7 (0xc2b bytes) 2024-04-24T18:08:47.606 INFO:teuthology.orchestra.run.smithi049.stdout:4840 write 0x2efe819 thru 0x2efebba (0x3a2 bytes) 2024-04-24T18:08:47.609 INFO:teuthology.orchestra.run.smithi049.stdout:4842 trunc from 0x51e3750 to 0x3980c78 2024-04-24T18:08:47.622 INFO:teuthology.orchestra.run.smithi049.stdout:4843 writesame 0x3dd9930 thru 0x3dde540 data_size 0x1(0x4c11 bytes) 2024-04-24T18:08:47.632 INFO:teuthology.orchestra.run.smithi049.stdout:4844 write 0x678ddfe thru 0x6795d66 (0x7f69 bytes) 2024-04-24T18:08:47.644 INFO:teuthology.orchestra.run.smithi049.stdout:4845 write 0xb001d1e thru 0xb0106af (0xe992 bytes) 2024-04-24T18:08:47.646 INFO:teuthology.orchestra.run.smithi049.stdout:4846 write 0x5e362dd thru 0x5e39b1d (0x3841 bytes) 2024-04-24T18:08:47.647 INFO:teuthology.orchestra.run.smithi049.stdout:4847 write 0x933905c thru 0x9343d15 (0xacba bytes) 2024-04-24T18:08:47.649 INFO:teuthology.orchestra.run.smithi049.stdout:4848 write 0x42db9c2 thru 0x42dbe45 (0x484 bytes) 2024-04-24T18:08:47.655 INFO:teuthology.orchestra.run.smithi049.stdout:4849 writesame 0xb54fd65 thru 0xb554e17 data_size 0x1(0x50b3 bytes) 2024-04-24T18:08:47.657 INFO:teuthology.orchestra.run.smithi049.stdout:4850 punch from 0x81ebeba to 0x81fa71c, (0xe862 bytes) 2024-04-24T18:08:47.657 INFO:teuthology.orchestra.run.smithi049.stdout:4851 write 0x6498fcf thru 0x649cc32 (0x3c64 bytes) 2024-04-24T18:08:47.658 INFO:teuthology.orchestra.run.smithi049.stdout:4852 writesame 0xc3ec54 thru 0xc42964 data_size 0x1b(0x3d11 bytes) 2024-04-24T18:08:47.660 INFO:teuthology.orchestra.run.smithi049.stdout:4853 read 0x1bb7aad thru 0x1bc0e03 (0x9357 bytes) 2024-04-24T18:08:47.665 INFO:teuthology.orchestra.run.smithi049.stdout:4854 trunc from 0xb554e18 to 0x7970b3b 2024-04-24T18:08:47.679 INFO:teuthology.orchestra.run.smithi049.stdout:4855 trunc from 0x7970b3b to 0x2489975 2024-04-24T18:08:47.689 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:47.697+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:47.707 INFO:teuthology.orchestra.run.smithi049.stdout:4856 writesame 0xee2a84c thru 0xee30425 data_size 0x3(0x5bda bytes) 2024-04-24T18:08:47.708 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:47.708 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:47.708 INFO:teuthology.orchestra.run.smithi049.stdout:4857 write 0x1609a49 thru 0x1615078 (0xb630 bytes) 2024-04-24T18:08:47.710 INFO:teuthology.orchestra.run.smithi049.stdout:4858 writesame 0x298132a thru 0x2981f5c data_size 0x3(0xc33 bytes) 2024-04-24T18:08:47.712 INFO:teuthology.orchestra.run.smithi049.stdout:4859 write 0xcfe0cfa thru 0xcfea87f (0x9b86 bytes) 2024-04-24T18:08:47.713 INFO:teuthology.orchestra.run.smithi049.stdout:4860 write 0x20513c0 thru 0x205c7c0 (0xb401 bytes) 2024-04-24T18:08:47.718 INFO:teuthology.orchestra.run.smithi049.stdout:4861 trunc from 0xee30426 to 0xc7ef35b 2024-04-24T18:08:47.722 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:47.727 INFO:teuthology.orchestra.run.smithi049.stdout:4862 write 0x7785c6f thru 0x778b6ec (0x5a7e bytes) 2024-04-24T18:08:47.729 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.734 INFO:teuthology.orchestra.run.smithi049.stdout:4866 trunc from 0xc7ef35b to 0x761e799 2024-04-24T18:08:47.748 INFO:teuthology.orchestra.run.smithi049.stdout:4867 trunc from 0x761e799 to 0xcc64cbe 2024-04-24T18:08:47.754 INFO:teuthology.orchestra.run.smithi049.stdout:4869 punch from 0x1ffb9c7 to 0x2008d48, (0xd381 bytes) 2024-04-24T18:08:47.754 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.754 INFO:teuthology.orchestra.run.smithi049.stdout:4871 read 0xc681d8b thru 0xc68e232 (0xc4a8 bytes) 2024-04-24T18:08:47.754 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:47.754 INFO:teuthology.orchestra.run.smithi049.stdout:4873 read 0x6558469 thru 0x6567f9d (0xfb35 bytes) 2024-04-24T18:08:47.755 INFO:teuthology.orchestra.run.smithi049.stdout:4874 clone 45 order 20 su 65536 sc 11 2024-04-24T18:08:47.790 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:47.798+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:47.860 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:08:47.866 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:47.867 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:47.880 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:47.920 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone44 from 0xcc64cbe (overlap 0x12b6926) to 0x10d783b 2024-04-24T18:08:47.920 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:47.927+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:47.992 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:48.001+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:48.015 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #43, image image_client.0-clone43 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent44 2024-04-24T18:08:48.026 INFO:teuthology.orchestra.run.smithi049.stdout:4876 punch from 0x5277feb to 0x5281985, (0x999a bytes) 2024-04-24T18:08:48.029 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:48.030 INFO:teuthology.orchestra.run.smithi049.stdout:4879 punch from 0x39c66d7 to 0x39c74d9, (0xe02 bytes) 2024-04-24T18:08:48.030 INFO:teuthology.orchestra.run.smithi049.stdout:4881 read 0x722b277 thru 0x7239482 (0xe20c bytes) 2024-04-24T18:08:48.030 INFO:teuthology.orchestra.run.smithi049.stdout:4883 write 0xb92121e thru 0xb92e420 (0xd203 bytes) 2024-04-24T18:08:48.034 INFO:teuthology.orchestra.run.smithi049.stdout:4884 write 0x1c91c0f thru 0x1c9e519 (0xc90b bytes) 2024-04-24T18:08:48.039 INFO:teuthology.orchestra.run.smithi049.stdout:4885 writesame 0xb117303 thru 0xb126c59 data_size 0x1(0xf957 bytes) 2024-04-24T18:08:48.061 INFO:teuthology.orchestra.run.smithi049.stdout:4886 write 0x40f8689 thru 0x40fd2bf (0x4c37 bytes) 2024-04-24T18:08:48.065 INFO:teuthology.orchestra.run.smithi049.stdout:4887 punch from 0x78565b2 to 0x7864539, (0xdf87 bytes) 2024-04-24T18:08:48.065 INFO:teuthology.orchestra.run.smithi049.stdout:4888 write 0x5644b2a thru 0x564c6ec (0x7bc3 bytes) 2024-04-24T18:08:48.068 INFO:teuthology.orchestra.run.smithi049.stdout:4890 clone 46 order 19 su 32768 sc 10 2024-04-24T18:08:48.070 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:48.070 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:48.080 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:48.125 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:48.133+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:48.195 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:48.203+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:48.211 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:48.211 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:48.220 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:48.222 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:48.300 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:48.307+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:48.368 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:48.387 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:48.392 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:48.392 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:48.392 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:48.392 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:48.393 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:48.400 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:48.408+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:48.403 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:48.406 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:48.425 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:48.445 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:48.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:48.485 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:48.485 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:48.485 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:48.485 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:48.486 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:48.486 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:48.486 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:48.486 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:48.486 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:48.488 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:48.488 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:48.503 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:48.511+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070434209999998, 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:08:48.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:08:48.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070416005999999, 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:08:48.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070394590999999, 2024-04-24T18:08:48.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:08:48.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.548 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070377440000001, 2024-04-24T18:08:48.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:08:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.550 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:08:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:08:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:08:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:08:48.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:08:48.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070361262999999, 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:08:48.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:08:48.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070343426000001, 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.560 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:08:48.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:08:48.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.564 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070312822999998, 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:48.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.568 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070292664, 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070276387, 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.573 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:08:48.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:08:48.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070258796000001, 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:48.581 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070242926999999, 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:48.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:08:48.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.585 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070224380999999, 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.586 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:08:48.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:08:48.589 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070208337, 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.590 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:08:48.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:08:48.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.594 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070191045000001, 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070169389, 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:08:48.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070150558999998, 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:08:48.603 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:08:48.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:08:48.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:08:48.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:08:48.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:08:48.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.608 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.608 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.608 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:48.609 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070134224, 2024-04-24T18:08:48.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:08:48.609 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.609 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:48.609 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.609 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.609 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:08:48.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.610 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:48.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:08:48.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:08:48.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:08:48.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:08:48.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:08:48.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:08:48.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.616 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070118398999998, 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:08:48.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.620 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.070099975000002, 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:08:48.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:08:48.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 22.0700836, 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.625 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:48.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:48.629 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:48.629 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:48.630 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:48.631 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:48.638+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:48.731 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:48.738+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:48.758 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:48.759 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:48.759 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:48.759 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:48.769 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:48.853 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:48.861+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:48.854 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:08:48.927 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone45 from 0xcc64cbe (overlap 0xcc64cbe) to 0x14707ac 2024-04-24T18:08:48.933 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:48.941+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:48.969 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:48.969 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:48.969 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:48.969 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:48.969 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:48.985 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:49.073 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:49.080+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:49.134 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:49.143+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:49.200 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:49.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070464506, 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:49.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:08:49.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:08:49.204 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070449965999998, 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.208 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070435115000002, 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:08:49.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:49.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:08:49.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:08:49.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070419966000003, 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.213 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070405180000002, 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.221 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070390590999999, 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:49.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:08:49.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:08:49.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070367765999997, 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070352112000002, 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.230 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:49.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:49.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:08:49.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070336990000001, 2024-04-24T18:08:49.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:08:49.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070321174, 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:08:49.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:08:49.242 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070306172000002, 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.243 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:49.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:08:49.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:08:49.246 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070291308000002, 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:08:49.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:08:49.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.250 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070270190999999, 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070254961000003, 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:49.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:08:49.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.070240224000003, 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:08:49.258 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:08:49.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.065139014000003, 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.065103168, 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:49.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:08:49.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:08:49.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.065083160999997, 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:49.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:08:49.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:08:49.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.063261969999999, 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.063236689999997, 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:08:49.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.282 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:49.283 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #44, image image_client.0-clone44 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent45 2024-04-24T18:08:49.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:4891 write 0x1bb47b6 thru 0x1bbe60f (0x9e5a bytes) 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:4892 punch from 0x2300ca6 to 0x2305647, (0x49a1 bytes) 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:4893 write 0xc499bcb thru 0xc4a256a (0x89a0 bytes) 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:4894 write 0x46c87cd thru 0x46d1aac (0x92e0 bytes) 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:4895 trunc from 0xcc64cbe to 0xa283b08 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:4897 read 0x4db713d thru 0x4dc350a (0xc3ce bytes) 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:4898 read 0x9ca8758 thru 0x9cb1ec8 (0x9771 bytes) 2024-04-24T18:08:49.285 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.286 INFO:teuthology.orchestra.run.smithi049.stdout:4901 read 0x382fe85 thru 0x383b040 (0xb1bc bytes) 2024-04-24T18:08:49.286 INFO:teuthology.orchestra.run.smithi049.stdout:4902 read 0x983e359 thru 0x9849d49 (0xb9f1 bytes) 2024-04-24T18:08:49.286 INFO:teuthology.orchestra.run.smithi049.stdout:4904 write 0x54dece7 thru 0x54ecc07 (0xdf21 bytes) 2024-04-24T18:08:49.286 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:49.292+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:49.292 INFO:teuthology.orchestra.run.smithi049.stdout:4905 writesame 0xa58a2a3 thru 0xa58e0ab data_size 0x1(0x3e09 bytes) 2024-04-24T18:08:49.293 INFO:teuthology.orchestra.run.smithi049.stdout:4906 punch from 0x5b5e7ce to 0x5b6db9e, (0xf3d0 bytes) 2024-04-24T18:08:49.293 INFO:teuthology.orchestra.run.smithi049.stdout:4907 write 0x40d2083 thru 0x40dd8d9 (0xb857 bytes) 2024-04-24T18:08:49.298 INFO:teuthology.orchestra.run.smithi049.stdout:4908 write 0x32878f thru 0x330a97 (0x8309 bytes) 2024-04-24T18:08:49.316 INFO:teuthology.orchestra.run.smithi049.stdout:4909 trunc from 0xa58e0ac to 0x27f5c60 2024-04-24T18:08:49.335 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.336 INFO:teuthology.orchestra.run.smithi049.stdout:4912 trunc from 0x27f5c60 to 0x1d079a7 2024-04-24T18:08:49.382 INFO:teuthology.orchestra.run.smithi049.stdout:4914 writesame 0xe752631 thru 0xe7572e1 data_size 0x1(0x4cb1 bytes) 2024-04-24T18:08:49.382 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.382 INFO:teuthology.orchestra.run.smithi049.stdout:4918 writesame 0x4ef40bd thru 0x4efc680 data_size 0xe(0x85c4 bytes) 2024-04-24T18:08:49.384 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:49.393+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:49.385 INFO:teuthology.orchestra.run.smithi049.stdout:4919 write 0x18753a8 thru 0x1881216 (0xbe6f bytes) 2024-04-24T18:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout:4921 read 0x2f57aab thru 0x2f637bf (0xbd15 bytes) 2024-04-24T18:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout:4923 read 0x8104bd2 thru 0x810dbd0 (0x8fff bytes) 2024-04-24T18:08:49.395 INFO:teuthology.orchestra.run.smithi049.stdout:4925 read 0xca67513 thru 0xca701e7 (0x8cd5 bytes) 2024-04-24T18:08:49.403 INFO:teuthology.orchestra.run.smithi049.stdout:4926 trunc from 0xe7572e2 to 0x721cc76 2024-04-24T18:08:49.419 INFO:teuthology.orchestra.run.smithi049.stdout:4927 writesame 0xac87bd0 thru 0xac908f0 data_size 0x1(0x8d21 bytes) 2024-04-24T18:08:49.419 INFO:teuthology.orchestra.run.smithi049.stdout:4928 read 0xa65f4f2 thru 0xa6609b6 (0x14c5 bytes) 2024-04-24T18:08:49.419 INFO:teuthology.orchestra.run.smithi049.stdout:4930 read 0xa47a017 thru 0xa480ab7 (0x6aa1 bytes) 2024-04-24T18:08:49.419 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.426 INFO:teuthology.orchestra.run.smithi049.stdout:4934 write 0xb96fbce thru 0xb97892a (0x8d5d bytes) 2024-04-24T18:08:49.431 INFO:teuthology.orchestra.run.smithi049.stdout:4935 read 0x576f31b thru 0x577b4d8 (0xc1be bytes) 2024-04-24T18:08:49.431 INFO:teuthology.orchestra.run.smithi049.stdout:4936 punch from 0x562dcc2 to 0x563864b, (0xa989 bytes) 2024-04-24T18:08:49.431 INFO:teuthology.orchestra.run.smithi049.stdout:4937 punch from 0x644a004 to 0x6458f46, (0xef42 bytes) 2024-04-24T18:08:49.431 INFO:teuthology.orchestra.run.smithi049.stdout:4939 read 0x461a7f1 thru 0x462a542 (0xfd52 bytes) 2024-04-24T18:08:49.431 INFO:teuthology.orchestra.run.smithi049.stdout:4942 punch from 0x1bc0131 to 0x1bc7ddd, (0x7cac bytes) 2024-04-24T18:08:49.432 INFO:teuthology.orchestra.run.smithi049.stdout:4943 write 0x9c153d5 thru 0x9c15775 (0x3a1 bytes) 2024-04-24T18:08:49.433 INFO:teuthology.orchestra.run.smithi049.stdout:4945 writesame 0x737dba0 thru 0x738ae0c data_size 0x1(0xd26d bytes) 2024-04-24T18:08:49.449 INFO:teuthology.orchestra.run.smithi049.stdout:4946 read 0x1516e12 thru 0x1520fc8 (0xa1b7 bytes) 2024-04-24T18:08:49.456 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:49.457 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:49.457 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:49.463 INFO:teuthology.orchestra.run.smithi049.stdout:4948 trunc from 0xb97892b to 0x138850d 2024-04-24T18:08:49.477 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:49.494 INFO:teuthology.orchestra.run.smithi049.stdout:4950 punch from 0xf33798 to 0xf3724b, (0x3ab3 bytes) 2024-04-24T18:08:49.494 INFO:teuthology.orchestra.run.smithi049.stdout:4951 read 0xfc7c90 thru 0xfc90eb (0x145c bytes) 2024-04-24T18:08:49.494 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:49.502+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:49.498 INFO:teuthology.orchestra.run.smithi049.stdout:4952 writesame 0x14b2e45 thru 0x14bb24c data_size 0x82(0x8408 bytes) 2024-04-24T18:08:49.515 INFO:teuthology.orchestra.run.smithi049.stdout:4953 write 0xb0fea3e thru 0xb107869 (0x8e2c bytes) 2024-04-24T18:08:49.522 INFO:teuthology.orchestra.run.smithi049.stdout:4955 write 0x286ad9a thru 0x286ee9a (0x4101 bytes) 2024-04-24T18:08:49.523 INFO:teuthology.orchestra.run.smithi049.stdout:4956 write 0x6091280 thru 0x6095cc6 (0x4a47 bytes) 2024-04-24T18:08:49.526 INFO:teuthology.orchestra.run.smithi049.stdout:4957 punch from 0x7739f87 to 0x773d306, (0x337f bytes) 2024-04-24T18:08:49.535 INFO:teuthology.orchestra.run.smithi049.stdout:4958 write 0xe8b5977 thru 0xe8bcc35 (0x72bf bytes) 2024-04-24T18:08:49.539 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.539 INFO:teuthology.orchestra.run.smithi049.stdout:4961 writesame 0xa9b5a70 thru 0xa9bb72a data_size 0x29(0x5cbb bytes) 2024-04-24T18:08:49.548 INFO:teuthology.orchestra.run.smithi049.stdout:4962 trunc from 0xe8bcc36 to 0x9389316 2024-04-24T18:08:49.560 INFO:teuthology.orchestra.run.smithi049.stdout:4963 read 0x383f8c8 thru 0x384d391 (0xdaca bytes) 2024-04-24T18:08:49.560 INFO:teuthology.orchestra.run.smithi049.stdout:4966 punch from 0x8ddbfdd to 0x8de274f, (0x6772 bytes) 2024-04-24T18:08:49.570 INFO:teuthology.orchestra.run.smithi049.stdout:4967 writesame 0xd0fad08 thru 0xd108abf data_size 0x84(0xddb8 bytes) 2024-04-24T18:08:49.570 INFO:teuthology.orchestra.run.smithi049.stdout:4968 punch from 0x98061c5 to 0x980fc9a, (0x9ad5 bytes) 2024-04-24T18:08:49.570 INFO:teuthology.orchestra.run.smithi049.stdout:4969 read 0xc53189a thru 0xc53a584 (0x8ceb bytes) 2024-04-24T18:08:49.570 INFO:teuthology.orchestra.run.smithi049.stdout:4970 write 0x4085eae thru 0x409079e (0xa8f1 bytes) 2024-04-24T18:08:49.575 INFO:teuthology.orchestra.run.smithi049.stdout:4971 writesame 0xcfdc913 thru 0xcfe1851 data_size 0x1(0x4f3f bytes) 2024-04-24T18:08:49.585 INFO:teuthology.orchestra.run.smithi049.stdout:4972 read 0x4b17948 thru 0x4b1c593 (0x4c4c bytes) 2024-04-24T18:08:49.585 INFO:teuthology.orchestra.run.smithi049.stdout:4973 trunc from 0xd108ac0 to 0xcfae08e 2024-04-24T18:08:49.586 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:49.594+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:49.592 INFO:teuthology.orchestra.run.smithi049.stdout:4974 read 0xa8cfe69 thru 0xa8d07c9 (0x961 bytes) 2024-04-24T18:08:49.592 INFO:teuthology.orchestra.run.smithi049.stdout:4976 writesame 0x23635a2 thru 0x2364cf9 data_size 0x18(0x1758 bytes) 2024-04-24T18:08:49.594 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.595 INFO:teuthology.orchestra.run.smithi049.stdout:4978 punch from 0x13fe27f to 0x140c445, (0xe1c6 bytes) 2024-04-24T18:08:49.595 INFO:teuthology.orchestra.run.smithi049.stdout:4980 punch from 0xcdd3c7 to 0xcea9e5, (0xd61e bytes) 2024-04-24T18:08:49.595 INFO:teuthology.orchestra.run.smithi049.stdout:4981 read 0xbdbf920 thru 0xbdca00a (0xa6eb bytes) 2024-04-24T18:08:49.595 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.595 INFO:teuthology.orchestra.run.smithi049.stdout:4987 punch from 0x12c54b1 to 0x12d0375, (0xaec4 bytes) 2024-04-24T18:08:49.595 INFO:teuthology.orchestra.run.smithi049.stdout:4988 write 0x1f6d340 thru 0x1f7c694 (0xf355 bytes) 2024-04-24T18:08:49.600 INFO:teuthology.orchestra.run.smithi049.stdout:4989 write 0x4a1aa8a thru 0x4a23b70 (0x90e7 bytes) 2024-04-24T18:08:49.604 INFO:teuthology.orchestra.run.smithi049.stdout:4991 punch from 0x9787d9b to 0x9789d19, (0x1f7e bytes) 2024-04-24T18:08:49.604 INFO:teuthology.orchestra.run.smithi049.stdout:4992 flatten 2024-04-24T18:08:49.650 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.650 INFO:teuthology.orchestra.run.smithi049.stdout:4995 write 0x381d4d0 thru 0x38225bc (0x50ed bytes) 2024-04-24T18:08:49.660 INFO:teuthology.orchestra.run.smithi049.stdout:4997 trunc from 0xcfae08e to 0x7c91cd6 2024-04-24T18:08:49.665 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:49.665 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:49.670 INFO:teuthology.orchestra.run.smithi049.stdout:4998 writesame 0x26f5d5d thru 0x26fb9da data_size 0x1(0x5c7e bytes) 2024-04-24T18:08:49.677 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:49.691 INFO:teuthology.orchestra.run.smithi049.stdout:4999 trunc from 0x7c91cd6 to 0xe0c089d 2024-04-24T18:08:49.691 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:49.699+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:49.696 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.696 INFO:teuthology.orchestra.run.smithi049.stdout:5001 writesame 0xb712968 thru 0xb71df86 data_size 0x1(0xb61f bytes) 2024-04-24T18:08:49.714 INFO:teuthology.orchestra.run.smithi049.stdout:5003 punch from 0xa08891 to 0xa106cd, (0x7e3c bytes) 2024-04-24T18:08:49.719 INFO:teuthology.orchestra.run.smithi049.stdout:5004 trunc from 0xe0c089d to 0x9b1c164 2024-04-24T18:08:49.729 INFO:teuthology.orchestra.run.smithi049.stdout:5005 read 0x36544ea thru 0x365d570 (0x9087 bytes) 2024-04-24T18:08:49.741 INFO:teuthology.orchestra.run.smithi049.stdout:5006 write 0xe790dd9 thru 0xe7919c8 (0xbf0 bytes) 2024-04-24T18:08:49.755 INFO:teuthology.orchestra.run.smithi049.stdout:5007 trunc from 0xe7919c9 to 0x4673a7d 2024-04-24T18:08:49.767 INFO:teuthology.orchestra.run.smithi049.stdout:5008 punch from 0x3b4b110 to 0x3b4c386, (0x1276 bytes) 2024-04-24T18:08:49.767 INFO:teuthology.orchestra.run.smithi049.stdout:5009 punch from 0x44c6770 to 0x44d3658, (0xcee8 bytes) 2024-04-24T18:08:49.770 INFO:teuthology.orchestra.run.smithi049.stdout:5010 trunc from 0x4673a7d to 0x69aad58 2024-04-24T18:08:49.775 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.775 INFO:teuthology.orchestra.run.smithi049.stdout:5015 read 0x468b043 thru 0x4691c15 (0x6bd3 bytes) 2024-04-24T18:08:49.776 INFO:teuthology.orchestra.run.smithi049.stdout:5017 punch from 0x20788c7 to 0x20881f0, (0xf929 bytes) 2024-04-24T18:08:49.776 INFO:teuthology.orchestra.run.smithi049.stdout:5018 punch from 0x4dfcb56 to 0x4e00121, (0x35cb bytes) 2024-04-24T18:08:49.776 INFO:teuthology.orchestra.run.smithi049.stdout:5020 trunc from 0x69aad58 to 0x64581b6 2024-04-24T18:08:49.781 INFO:teuthology.orchestra.run.smithi049.stdout:5021 punch from 0x47d1220 to 0x47e0f54, (0xfd34 bytes) 2024-04-24T18:08:49.781 INFO:teuthology.orchestra.run.smithi049.stdout:5023 read 0xcba230 thru 0xcbba10 (0x17e1 bytes) 2024-04-24T18:08:49.781 INFO:teuthology.orchestra.run.smithi049.stdout:5024 read 0x4d592d6 thru 0x4d5e1f0 (0x4f1b bytes) 2024-04-24T18:08:49.787 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:49.796+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:49.790 INFO:teuthology.orchestra.run.smithi049.stdout:5025 trunc from 0x64581b6 to 0xe59fb5e 2024-04-24T18:08:49.796 INFO:teuthology.orchestra.run.smithi049.stdout:5026 write 0x7d84316 thru 0x7d87ebf (0x3baa bytes) 2024-04-24T18:08:49.799 INFO:teuthology.orchestra.run.smithi049.stdout:5028 writesame 0xbbf12d2 thru 0xbbf4421 data_size 0x18(0x3150 bytes) 2024-04-24T18:08:49.801 INFO:teuthology.orchestra.run.smithi049.stdout:5029 read 0x3b6c594 thru 0x3b713b1 (0x4e1e bytes) 2024-04-24T18:08:49.813 INFO:teuthology.orchestra.run.smithi049.stdout:5031 trunc from 0xe59fb5e to 0x3b3eeb8 2024-04-24T18:08:49.827 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.827 INFO:teuthology.orchestra.run.smithi049.stdout:5033 write 0xadfb95 thru 0xaeb452 (0xb8be bytes) 2024-04-24T18:08:49.829 INFO:teuthology.orchestra.run.smithi049.stdout:5034 read 0x2560dc0 thru 0x256fd11 (0xef52 bytes) 2024-04-24T18:08:49.829 INFO:teuthology.orchestra.run.smithi049.stdout:5035 punch from 0x2b743d0 to 0x2b7724f, (0x2e7f bytes) 2024-04-24T18:08:49.843 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.672455415999998, 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.844 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.652816289999997, 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.848 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:49.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:49.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.652784568000001, 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.852 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:49.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:49.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.652761151, 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:49.856 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:49.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:49.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:49.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.652731078000002, 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.860 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:49.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:49.864 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651685931000003, 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.868 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651668700000002, 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:49.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.872 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651651702000002, 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:49.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:49.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.876 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651633021000002, 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651615552999999, 2024-04-24T18:08:49.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:49.884 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651599429000001, 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:49.888 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651568322999999, 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651549121999999, 2024-04-24T18:08:49.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.651528079999999, 2024-04-24T18:08:49.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.650773131999998, 2024-04-24T18:08:49.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:49.904 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.65074663, 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.908 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.6507291, 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:49.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:49.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.912 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.649265939999999, 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:49.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:49.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.916 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.637427291999998, 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:49.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 37.637410949, 2024-04-24T18:08:49.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:49.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:49.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:49.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:49.924 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:49.925 INFO:teuthology.orchestra.run.smithi049.stdout:5036 write 0xc0dd10d thru 0xc0e52de (0x81d2 bytes) 2024-04-24T18:08:49.925 INFO:teuthology.orchestra.run.smithi049.stdout:5038 write 0x111ec79 thru 0x112d32d (0xe6b5 bytes) 2024-04-24T18:08:49.925 INFO:teuthology.orchestra.run.smithi049.stdout:5039 read 0x17a646f thru 0x17b487a (0xe40c bytes) 2024-04-24T18:08:49.925 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.926 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:49.928 INFO:teuthology.orchestra.run.smithi049.stdout:5044 write 0xe80d24e thru 0xe818e99 (0xbc4c bytes) 2024-04-24T18:08:49.928 INFO:teuthology.orchestra.run.smithi049.stdout:5045 read 0x4864a2c thru 0x486c17d (0x7752 bytes) 2024-04-24T18:08:49.928 INFO:teuthology.orchestra.run.smithi049.stdout:5046 trunc from 0xe818e9a to 0x691bda3 2024-04-24T18:08:49.928 INFO:teuthology.orchestra.run.smithi049.stdout:5047 write 0x2302eb thru 0x236f7d (0x6c93 bytes) 2024-04-24T18:08:49.928 INFO:teuthology.orchestra.run.smithi049.stdout:5048 read 0x65e58f9 thru 0x65e6f67 (0x166f bytes) 2024-04-24T18:08:49.928 INFO:teuthology.orchestra.run.smithi049.stdout:5050 read 0x3596305 thru 0x35a16dd (0xb3d9 bytes) 2024-04-24T18:08:49.928 INFO:teuthology.orchestra.run.smithi049.stdout:5051 writesame 0x7174dd5 thru 0x71848c3 data_size 0x17(0xfaef bytes) 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:5052 write 0xc1d4690 thru 0xc1e3b12 (0xf483 bytes) 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:5053 read 0x386badb thru 0x3874970 (0x8e96 bytes) 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:5055 read 0x3c9db91 thru 0x3caaf0f (0xd37f bytes) 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:5056 punch from 0x6560ea2 to 0x6561601, (0x75f bytes) 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:5060 writesame 0x8cd7ac8 thru 0x8ce3bc4 data_size 0x29(0xc0fd bytes) 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:5062 read 0x6a1ef7b thru 0x6a2c5fb (0xd681 bytes) 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.929 INFO:teuthology.orchestra.run.smithi049.stdout:5064 trunc from 0xc1e3b13 to 0x478b9b8 2024-04-24T18:08:49.930 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:49.934+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:49.934 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:49.934 INFO:teuthology.orchestra.run.smithi049.stdout:5068 read 0x2bb57a6 thru 0x2bbf757 (0x9fb2 bytes) 2024-04-24T18:08:49.946 INFO:teuthology.orchestra.run.smithi049.stdout:5069 write 0x9831c10 thru 0x983d020 (0xb411 bytes) 2024-04-24T18:08:49.950 INFO:teuthology.orchestra.run.smithi049.stdout:5070 write 0x84fd40f thru 0x85009b3 (0x35a5 bytes) 2024-04-24T18:08:49.962 INFO:teuthology.orchestra.run.smithi049.stdout:5071 write 0xa85a84a thru 0xa863d70 (0x9527 bytes) 2024-04-24T18:08:49.972 INFO:teuthology.orchestra.run.smithi049.stdout:5072 trunc from 0xa863d71 to 0x42ac4d6 2024-04-24T18:08:49.983 INFO:teuthology.orchestra.run.smithi049.stdout:5073 read 0x6787cb thru 0x686a63 (0xe299 bytes) 2024-04-24T18:08:49.983 INFO:teuthology.orchestra.run.smithi049.stdout:5074 write 0x1880f04 thru 0x188c654 (0xb751 bytes) 2024-04-24T18:08:50.003 INFO:teuthology.orchestra.run.smithi049.stdout:5075 write 0xc5d9339 thru 0xc5e4ec9 (0xbb91 bytes) 2024-04-24T18:08:50.007 INFO:teuthology.orchestra.run.smithi049.stdout:5078 write 0x4effe29 thru 0x4f0905c (0x9234 bytes) 2024-04-24T18:08:50.011 INFO:teuthology.orchestra.run.smithi049.stdout:5080 write 0xc073e8e thru 0xc07cb32 (0x8ca5 bytes) 2024-04-24T18:08:50.014 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:50.022 INFO:teuthology.orchestra.run.smithi049.stdout:5084 trunc from 0xc5e4eca to 0x5698759 2024-04-24T18:08:50.026 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:50.035+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:50.043 INFO:teuthology.orchestra.run.smithi049.stdout:5085 write 0x870902f thru 0x87182dc (0xf2ae bytes) 2024-04-24T18:08:50.048 INFO:teuthology.orchestra.run.smithi049.stdout:5086 write 0x79f1c48 thru 0x79f1f29 (0x2e2 bytes) 2024-04-24T18:08:50.052 INFO:teuthology.orchestra.run.smithi049.stdout:5088 write 0x5c8cbc2 thru 0x5c8ead1 (0x1f10 bytes) 2024-04-24T18:08:50.056 INFO:teuthology.orchestra.run.smithi049.stdout:5092 trunc from 0x87182dd to 0x9f1b47d 2024-04-24T18:08:50.060 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:50.060 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:50.060 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:50.060 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:50.062 INFO:teuthology.orchestra.run.smithi049.stdout:5094 write 0x8fef312 thru 0x8ffb0d8 (0xbdc7 bytes) 2024-04-24T18:08:50.067 INFO:teuthology.orchestra.run.smithi049.stdout:5095 read 0x7e8e7b3 thru 0x7e914c9 (0x2d17 bytes) 2024-04-24T18:08:50.067 INFO:teuthology.orchestra.run.smithi049.stdout:5096 read 0x5913e30 thru 0x591fc17 (0xbde8 bytes) 2024-04-24T18:08:50.067 INFO:teuthology.orchestra.run.smithi049.stdout:5097 write 0x321d1f9 thru 0x322494f (0x7757 bytes) 2024-04-24T18:08:50.071 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:50.079 INFO:teuthology.orchestra.run.smithi049.stdout:5098 write 0xbaf6907 thru 0xbb01ff5 (0xb6ef bytes) 2024-04-24T18:08:50.092 INFO:teuthology.orchestra.run.smithi049.stdout:5099 trunc from 0xbb01ff6 to 0x467bfb5 2024-04-24T18:08:50.116 INFO:teuthology.orchestra.run.smithi049.stdout:5100 write 0x90b70b9 thru 0x90bd917 (0x685f bytes) 2024-04-24T18:08:50.126 INFO:teuthology.orchestra.run.smithi049.stdout:5101 trunc from 0x90bd918 to 0xe6ce509 2024-04-24T18:08:50.127 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:50.135+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:50.132 INFO:teuthology.orchestra.run.smithi049.stdout:5102 write 0x209bbda thru 0x20a440c (0x8833 bytes) 2024-04-24T18:08:50.138 INFO:teuthology.orchestra.run.smithi049.stdout:5103 write 0x79892ac thru 0x798a79b (0x14f0 bytes) 2024-04-24T18:08:50.138 INFO:teuthology.orchestra.run.smithi049.stdout:5104 punch from 0x93cb3f4 to 0x93ce79c, (0x33a8 bytes) 2024-04-24T18:08:50.138 INFO:teuthology.orchestra.run.smithi049.stdout:5105 writesame 0x9690ec3 thru 0x969b49e data_size 0x37(0xa5dc bytes) 2024-04-24T18:08:50.142 INFO:teuthology.orchestra.run.smithi049.stdout:5107 read 0x55e4785 thru 0x55e94aa (0x4d26 bytes) 2024-04-24T18:08:50.142 INFO:teuthology.orchestra.run.smithi049.stdout:5108 read 0x56d722 thru 0x578d19 (0xb5f8 bytes) 2024-04-24T18:08:50.143 INFO:teuthology.orchestra.run.smithi049.stdout:5109 punch from 0xe1f2046 to 0xe1f8836, (0x67f0 bytes) 2024-04-24T18:08:50.143 INFO:teuthology.orchestra.run.smithi049.stdout:5110 write 0x4857ef7 thru 0x48660d2 (0xe1dc bytes) 2024-04-24T18:08:50.148 INFO:teuthology.orchestra.run.smithi049.stdout:5111 write 0x6177e76 thru 0x618622e (0xe3b9 bytes) 2024-04-24T18:08:50.152 INFO:teuthology.orchestra.run.smithi049.stdout:5112 write 0x5cfe2fa thru 0x5d01d4d (0x3a54 bytes) 2024-04-24T18:08:50.156 INFO:teuthology.orchestra.run.smithi049.stdout:5113 read 0x1676abe thru 0x1677030 (0x573 bytes) 2024-04-24T18:08:50.163 INFO:teuthology.orchestra.run.smithi049.stdout:5114 write 0xee62459 thru 0xee62515 (0xbd bytes) 2024-04-24T18:08:50.166 INFO:teuthology.orchestra.run.smithi049.stdout:5115 writesame 0xdcf615e thru 0xdd05f00 data_size 0x1(0xfda3 bytes) 2024-04-24T18:08:50.198 INFO:teuthology.orchestra.run.smithi049.stdout:5117 trunc from 0xee62516 to 0x1206b62 2024-04-24T18:08:50.218 INFO:teuthology.orchestra.run.smithi049.stdout:5119 read 0x786311 thru 0x789417 (0x3107 bytes) 2024-04-24T18:08:50.219 INFO:teuthology.orchestra.run.smithi049.stdout:5120 read 0xc37f0f thru 0xc3ba56 (0x3b48 bytes) 2024-04-24T18:08:50.227 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:50.235+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:50.241 INFO:teuthology.orchestra.run.smithi049.stdout:5121 writesame 0xddfe342 thru 0xde007fd data_size 0x2(0x24bc bytes) 2024-04-24T18:08:50.241 INFO:teuthology.orchestra.run.smithi049.stdout:5125 punch from 0x2010d7a to 0x2017e66, (0x70ec bytes) 2024-04-24T18:08:50.241 INFO:teuthology.orchestra.run.smithi049.stdout:5126 write 0x8c778f9 thru 0x8c85bb6 (0xe2be bytes) 2024-04-24T18:08:50.246 INFO:teuthology.orchestra.run.smithi049.stdout:5127 punch from 0xc3f72ba to 0xc3fb27c, (0x3fc2 bytes) 2024-04-24T18:08:50.246 INFO:teuthology.orchestra.run.smithi049.stdout:5130 write 0xda3659a thru 0xda397e4 (0x324b bytes) 2024-04-24T18:08:50.249 INFO:teuthology.orchestra.run.smithi049.stdout:5131 punch from 0xc755c1f to 0xc764ecc, (0xf2ad bytes) 2024-04-24T18:08:50.250 INFO:teuthology.orchestra.run.smithi049.stdout:5133 punch from 0x5aa27cd to 0x5aaa133, (0x7966 bytes) 2024-04-24T18:08:50.250 INFO:teuthology.orchestra.run.smithi049.stdout:5134 read 0x5750526 thru 0x575136d (0xe48 bytes) 2024-04-24T18:08:50.250 INFO:teuthology.orchestra.run.smithi049.stdout:5139 write 0x6e2ea1e thru 0x6e3c50a (0xdaed bytes) 2024-04-24T18:08:50.254 INFO:teuthology.orchestra.run.smithi049.stdout:5141 read 0x560e1ce thru 0x561965b (0xb48e bytes) 2024-04-24T18:08:50.254 INFO:teuthology.orchestra.run.smithi049.stdout:5142 read 0x679f104 thru 0x67a2545 (0x3442 bytes) 2024-04-24T18:08:50.254 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:50.255 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:50.255 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:50.255 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:50.255 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:50.258 INFO:teuthology.orchestra.run.smithi049.stdout:5143 trunc from 0xde007fe to 0xa53564e 2024-04-24T18:08:50.270 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:50.276 INFO:teuthology.orchestra.run.smithi049.stdout:5144 writesame 0xbb8c5ae thru 0xbb97b6d data_size 0x20(0xb5c0 bytes) 2024-04-24T18:08:50.276 INFO:teuthology.orchestra.run.smithi049.stdout:5145 read 0x36efac1 thru 0x36f00e9 (0x629 bytes) 2024-04-24T18:08:50.279 INFO:teuthology.orchestra.run.smithi049.stdout:5146 trunc from 0xbb97b6e to 0x91547f8 2024-04-24T18:08:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:50.285 INFO:teuthology.orchestra.run.smithi049.stdout:5149 trunc from 0x91547f8 to 0x87cefa2 2024-04-24T18:08:50.295 INFO:teuthology.orchestra.run.smithi049.stdout:5150 trunc from 0x87cefa2 to 0x8688c36 2024-04-24T18:08:50.300 INFO:teuthology.orchestra.run.smithi049.stdout:5151 read 0x4cde904 thru 0x4ceb81b (0xcf18 bytes) 2024-04-24T18:08:50.300 INFO:teuthology.orchestra.run.smithi049.stdout:5152 writesame 0x2c185d0 thru 0x2c197f9 data_size 0xf(0x122a bytes) 2024-04-24T18:08:50.308 INFO:teuthology.orchestra.run.smithi049.stdout:5153 write 0x8df282d thru 0x8df9f00 (0x76d4 bytes) 2024-04-24T18:08:50.312 INFO:teuthology.orchestra.run.smithi049.stdout:5154 read 0x277496c thru 0x2775393 (0xa28 bytes) 2024-04-24T18:08:50.312 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:50.312 INFO:teuthology.orchestra.run.smithi049.stdout:5156 writesame 0x17c263 thru 0x17dd36 data_size 0x4(0x1ad4 bytes) 2024-04-24T18:08:50.315 INFO:teuthology.orchestra.run.smithi049.stdout:5157 read 0x4e19c7c thru 0x4e20def (0x7174 bytes) 2024-04-24T18:08:50.315 INFO:teuthology.orchestra.run.smithi049.stdout:5160 read 0x4665dff thru 0x4670b2c (0xad2e bytes) 2024-04-24T18:08:50.315 INFO:teuthology.orchestra.run.smithi049.stdout:5161 write 0x8cfdaad thru 0x8d03cf3 (0x6247 bytes) 2024-04-24T18:08:50.330 INFO:teuthology.orchestra.run.smithi049.stdout:5162 write 0xe435455 thru 0xe440aeb (0xb697 bytes) 2024-04-24T18:08:50.335 INFO:teuthology.orchestra.run.smithi049.stdout:5163 punch from 0x75b56bc to 0x75bb4fc, (0x5e40 bytes) 2024-04-24T18:08:50.335 INFO:teuthology.orchestra.run.smithi049.stdout:5165 writesame 0x2ceb7cb thru 0x2cebeb9 data_size 0x1(0x6ef bytes) 2024-04-24T18:08:50.338 INFO:teuthology.orchestra.run.smithi049.stdout:5166 clone 47 order 18 su 2048 sc 2 2024-04-24T18:08:50.352 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:50.361+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:50.429 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:50.438+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:50.436 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.104247430000001, 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:50.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.439 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.104231523999999, 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.441 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.443 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.104179557000002, 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:08:50.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:08:50.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.448 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.103256082999998, 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:08:50.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.103239451, 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.453 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11809 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2965504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.102555998, 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042364339000000001, 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.457 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11809 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340721+0000", 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1370000000000002e-06 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340727+0000", 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.533e-06 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340732+0000", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5329999999999996e-06 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382924+0000", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042192039000000001 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383012+0000", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7669000000000004e-05 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383083+0000", 2024-04-24T18:08:50.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0988999999999994e-05 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.102537847999997, 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.461 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:08:50.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:08:50.464 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.102521346000003, 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:08:50.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:08:50.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:08:50.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.468 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.101618479999999, 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.469 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:50.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:08:50.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.472 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.101610454999999, 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.476 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.100546313000002, 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.477 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.480 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.100530343999999, 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.040331904999999, 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.485 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:50.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:08:50.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:08:50.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:08:50.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136785 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 215040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.871322816999999, 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430404999999997, 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.490 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136785 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:50.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571953+0000", 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.08e-06 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571959+0000", 2024-04-24T18:08:50.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3469999999999998e-06 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571962+0000", 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8079999999999998e-06 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614268+0000", 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042306032 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614315+0000", 2024-04-24T18:08:50.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6529999999999997e-05 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614382+0000", 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6915000000000006e-05 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.871307872999999, 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:08:50.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:08:50.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.871291639000001, 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:08:50.498 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:50.502 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.871276419000001, 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:50.506 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.87124884, 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:08:50.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:08:50.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.510 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.871230539000001, 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:50.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:08:50.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:08:50.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.87121424, 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:50.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:50.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:08:50.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.518 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:50.519 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:50.519 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:50.519 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:50.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:50.564 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:50.573+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:50.630 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:50.639+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:50.665 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:50.683 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:50.688 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:50.688 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:50.699 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:50.701 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:50.720 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:50.731 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:50.740+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:50.738 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:50.758 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:Error ENXIO: problem getting command descriptions from osd.6 2024-04-24T18:08:50.777 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:50.779 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-04-24T18:08:50.780 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-04-24T18:08:50.824 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:50.824 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:50.831 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:50.840+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:50.834 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:50.869 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:08:50.932 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:50.941+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:50.969 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:50.969 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:50.978 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:50.987+0000 7f6edd7a9540 -1 osd.6 67 log_to_monitors true 2024-04-24T18:08:50.983 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:51.068 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:51.076+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:51.071 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #45, image image_client.0-clone45 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent46 2024-04-24T18:08:51.106 INFO:teuthology.orchestra.run.smithi049.stdout:5167 read 0xb4ce811 thru 0xb4cf1d6 (0x9c6 bytes) 2024-04-24T18:08:51.106 INFO:teuthology.orchestra.run.smithi049.stdout:5168 read 0x3fd20ba thru 0x3fd709b (0x4fe2 bytes) 2024-04-24T18:08:51.106 INFO:teuthology.orchestra.run.smithi049.stdout:5171 read 0xc37eba thru 0xc4650d (0xe654 bytes) 2024-04-24T18:08:51.107 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:51.107 INFO:teuthology.orchestra.run.smithi049.stdout:5174 punch from 0xaaa53f to 0xaab196, (0xc57 bytes) 2024-04-24T18:08:51.111 INFO:teuthology.orchestra.run.smithi049.stdout:5175 write 0xb139a29 thru 0xb13f617 (0x5bef bytes) 2024-04-24T18:08:51.115 INFO:teuthology.orchestra.run.smithi049.stdout:5177 punch from 0xcacb862 to 0xcad640f, (0xabad bytes) 2024-04-24T18:08:51.116 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:51.116 INFO:teuthology.orchestra.run.smithi049.stdout:5179 writesame 0x13c4a3d thru 0x13d3cae data_size 0x1(0xf272 bytes) 2024-04-24T18:08:51.134 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:51.142+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:51.141 INFO:teuthology.orchestra.run.smithi049.stdout:5180 writesame 0xd6ec07c thru 0xd6ecec3 data_size 0x4(0xe48 bytes) 2024-04-24T18:08:51.154 INFO:teuthology.orchestra.run.smithi049.stdout:5182 write 0xede24e8 thru 0xede6211 (0x3d2a bytes) 2024-04-24T18:08:51.160 INFO:teuthology.orchestra.run.smithi049.stdout:5183 punch from 0xd6918a1 to 0xd69524d, (0x39ac bytes) 2024-04-24T18:08:51.160 INFO:teuthology.orchestra.run.smithi049.stdout:5184 punch from 0x6f0ce20 to 0x6f16421, (0x9601 bytes) 2024-04-24T18:08:51.160 INFO:teuthology.orchestra.run.smithi049.stdout:5185 write 0xbf90ba9 thru 0xbf99ad3 (0x8f2b bytes) 2024-04-24T18:08:51.164 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:51.164 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:51.164 INFO:teuthology.orchestra.run.smithi049.stdout:5186 write 0x7b8e263 thru 0x7b9716b (0x8f09 bytes) 2024-04-24T18:08:51.168 INFO:teuthology.orchestra.run.smithi049.stdout:5187 clone 48 order 22 su 32768 sc 13 2024-04-24T18:08:51.175 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:51.259 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:51.267+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:51.336 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:51.344+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:51.338 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:51.338 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:51.348 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:51.437 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:51.446+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:51.513 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:51.513 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:51.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:51.538 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:51.547+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:51.640 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:51.648+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:51.655 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:51.655 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:51.665 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:51.740 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:51.749+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:51.793 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:51.793 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:51.803 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:51.841 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:51.850+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:51.863 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:08:51.875 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:51.884+0000 7f6ecd1bb700 -1 osd.6 67 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-24T18:08:51.919 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone47 from 0xede6212 (overlap 0xe440aec) to 0x2da6895 2024-04-24T18:08:51.942 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:51.951+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:52.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:52.036 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:52.036 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:52.045 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:52.057 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:52.065+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:52.144 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:52.153+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:52.178 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:52.178 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:52.178 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:52.178 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:52.192 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-24T18:08:52.246 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:52.254+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:52.310 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:52.311 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:52.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:52.346 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:52.355+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:52.447 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:52.455+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:52.548 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:52.556+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:52.548 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:52.548 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:52.559 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:52.562 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #46, image image_client.0-clone46 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent47 2024-04-24T18:08:52.649 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:52.658+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:52.711 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:52.711 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:52.723 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:52.764 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:52.773+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:52.781 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:52.834 INFO:teuthology.orchestra.run.smithi049.stdout:5188 writesame 0xbbd570e thru 0xbbe31a3 data_size 0x31(0xda96 bytes) 2024-04-24T18:08:52.850 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:52.859+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:52.875 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.0 to 1 (10000) 2024-04-24T18:08:52.889 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-24T18:08:52.907 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:52.907 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:52.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:52.951 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:52.959+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:52.987 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:53.006 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:53.025 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:53.046 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:53.051 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:53.060+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:53.066 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:53.085 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:53.094 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:53.094 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:53.094 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:53.094 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:53.094 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:53.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:53.105 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:53.123 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:53.125 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_delete_sleep = '' osd_delete_sleep_hdd = '' osd_delete_sleep_hybrid = '' osd_delete_sleep_ssd = '' osd_enable_op_tracker = '' osd_max_backfills = '' osd_recovery_max_active = '' osd_recovery_max_active_hdd = '' osd_recovery_max_active_ssd = '' osd_recovery_sleep = '' osd_recovery_sleep_hdd = '' osd_recovery_sleep_hybrid = '' osd_recovery_sleep_ssd = '' osd_scrub_sleep = '' osd_snap_trim_sleep = '' osd_snap_trim_sleep_hdd = '' osd_snap_trim_sleep_hybrid = '' osd_snap_trim_sleep_ssd = '' 2024-04-24T18:08:53.126 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:53.152 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:53.160+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:08:53.252 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:53.261+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:53.253 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:53.253 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123957324999999, 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:08:53.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:08:53.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123942784999997, 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123927934000001, 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:08:53.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123912785000002, 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:08:53.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123897999, 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:53.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:53.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.273 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123883409999998, 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:08:53.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123860585000003, 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.282 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123844931000001, 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.286 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123829809, 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.287 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:08:53.290 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123813992999999, 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:08:53.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:53.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123798991000001, 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.295 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:08:53.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:08:53.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123784127, 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:08:53.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:08:53.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123763009999998, 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.306 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123747780000002, 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.307 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:08:53.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.123733043000001, 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.311 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.314 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.118631833000002, 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:08:53.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.118595986999999, 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.322 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.118575980000003, 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.323 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:08:53.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:08:53.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.116754788999998, 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.327 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:08:53.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.116729509000002, 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:08:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:08:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.335 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.335 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:53.336 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:53.377 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:53.386+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:53.454 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:53.463+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:53.505 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:53.505 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:53.505 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:53.505 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:53.516 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:53.559 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:53.568+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:53.645 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:53.645 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:53.645 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:53.645 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:53.645 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:53.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:53.656 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:53.665+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:08:53.757 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:53.766+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.655418908999998, 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:53.831 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.635779782999997, 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.835 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.635748061000001, 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.836 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:53.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:53.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:53.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.635724644, 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.840 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:53.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:53.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.635694571000002, 2024-04-24T18:08:53.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.848 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634649424000003, 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:53.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:53.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.852 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634632193000002, 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:53.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:53.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:53.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634615195000002, 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:53.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634596514000002, 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.861 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634579045999999, 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.865 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634562922000001, 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.869 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:53.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:53.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:53.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634531815999999, 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:53.873 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.877 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634512614999998, 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:53.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:53.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.634491572999998, 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:53.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:53.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.633736624999997, 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.633710123, 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:53.890 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.633692592999999, 2024-04-24T18:08:53.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.898 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.632229432999999, 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:53.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:53.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:53.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.620390784999998, 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.906 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 41.620374441999999, 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.907 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:53.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:53.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:53.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:53.911 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:53.912 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:08:53.913 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:53.914 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.2 to 1 (10000) 2024-04-24T18:08:53.915 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-24T18:08:53.915 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:53.921+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:54.013 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:54.022+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4.d49066b8 (undecoded) ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.176211004, 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.17623618899999999, 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "waiting for peered", 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.048 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:08:54.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:08:54.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:08:54.052 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.052 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.052 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.052 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.052 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-24T18:08:54.052 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 1 2024-04-24T18:08:54.052 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:54.058 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:54.114 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:54.123+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:54.183 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:54.183 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:54.183 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:54.183 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:54.183 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:54.192 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:54.235 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:54.244+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:54.317 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:54.326+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.985662652000002, 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:08:54.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:08:54.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.322 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.985646746, 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:54.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.985594778999996, 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.327 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:08:54.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:08:54.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.984671305000006, 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.331 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.984654672999994, 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.335 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:08:54.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11809 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2965504~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.983971220000001, 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042364339000000001, 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.339 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11809 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340718+0000", 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340721+0000", 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1370000000000002e-06 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340727+0000", 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.533e-06 2024-04-24T18:08:54.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340732+0000", 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5329999999999996e-06 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382924+0000", 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042192039000000001 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383012+0000", 2024-04-24T18:08:54.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7669000000000004e-05 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383083+0000", 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.0988999999999994e-05 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.983953069999998, 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:08:54.343 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.983936568000004, 2024-04-24T18:08:54.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:54.351 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.983033702, 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:08:54.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.355 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.983025677000001, 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:08:54.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.981961534999996, 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:54.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:08:54.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.981945565999993, 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:08:54.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.921747127, 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:54.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136785 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 215040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.752738039, 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430404999999997, 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.373 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136785 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571953+0000", 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.08e-06 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571959+0000", 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3469999999999998e-06 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571962+0000", 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8079999999999998e-06 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614268+0000", 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042306032 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614315+0000", 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6529999999999997e-05 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614382+0000", 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6915000000000006e-05 2024-04-24T18:08:54.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.752723095, 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.377 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:54.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:08:54.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:08:54.380 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.752706861, 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.381 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.752691640999998, 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:08:54.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.752664062000001, 2024-04-24T18:08:54.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:54.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:08:54.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.752645761, 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:08:54.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:08:54.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.395 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.752629462000002, 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.396 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:08:54.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:08:54.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.399 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:54.400 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:54.442 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:54.450+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:54.519 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:54.528+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:54.565 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:54.565 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:54.565 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:54.565 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:54.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:54.620 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:54.629+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:54.697 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:54.697 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:54.697 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:54.697 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:54.697 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:54.707 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:54.750 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:54.759+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:54.822 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:54.830+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:54.831 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:54.831 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:54.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:54.831 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:54.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.831 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.831 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474928282, 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:54.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:08:54.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474920152999999, 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.47491213, 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474903217000001, 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.846 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474894398, 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474885266000001, 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.854 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:08:54.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474065007, 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.858 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474037807999999, 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.862 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474020337999999, 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.866 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.474004998000002, 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.870 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473980919999999, 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.874 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:54.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:08:54.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473965355000001, 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:08:54.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:08:54.881 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473950393999999, 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:08:54.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:08:54.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:08:54.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:08:54.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473933455000001, 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473917050000001, 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.890 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473900611000001, 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.894 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:08:54.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473885770999999, 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.898 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473869347000001, 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:08:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:08:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473852132000001, 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:08:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 24.473834666999998, 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:08:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:54.915 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:54.916 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:08:54.917 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:54.918 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.3 to 1 (10000) 2024-04-24T18:08:54.918 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-24T18:08:54.959 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:54.967+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:54.971 INFO:teuthology.orchestra.run.smithi049.stdout:5190 write 0x3a4419f thru 0x3a4fcab (0xbb0d bytes) 2024-04-24T18:08:55.002 INFO:teuthology.orchestra.run.smithi049.stdout:5191 writesame 0xb0de8c4 thru 0xb0e148b data_size 0xc(0x2bc8 bytes) 2024-04-24T18:08:55.005 INFO:teuthology.orchestra.run.smithi049.stdout:5192 writesame 0x4bde0d4 thru 0x4beba57 data_size 0x2(0xd984 bytes) 2024-04-24T18:08:55.024 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:55.032+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:55.048 INFO:teuthology.orchestra.run.smithi049.stdout:5193 write 0x3b67bb5 thru 0x3b725dc (0xaa28 bytes) 2024-04-24T18:08:55.091 INFO:teuthology.orchestra.run.smithi049.stdout:5194 read 0xd90a18d thru 0xd90b849 (0x16bd bytes) 2024-04-24T18:08:55.091 INFO:teuthology.orchestra.run.smithi049.stdout:5195 writesame 0x63c11ad thru 0x63cc4cf data_size 0xb(0xb323 bytes) 2024-04-24T18:08:55.101 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:55.101 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:55.101 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:55.101 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:55.114 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:55.115 INFO:teuthology.orchestra.run.smithi049.stdout:5196 read 0xc2cbb4e thru 0xc2d1f6e (0x6421 bytes) 2024-04-24T18:08:55.115 INFO:teuthology.orchestra.run.smithi049.stdout:5198 punch from 0xebd6ab0 to 0xebe03ca, (0x991a bytes) 2024-04-24T18:08:55.115 INFO:teuthology.orchestra.run.smithi049.stdout:5200 read 0x9f07a19 thru 0x9f167cc (0xedb4 bytes) 2024-04-24T18:08:55.119 INFO:teuthology.orchestra.run.smithi049.stdout:5201 trunc from 0xede6212 to 0xc1f429d 2024-04-24T18:08:55.125 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:55.133+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:55.137 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:08:55.211 INFO:teuthology.orchestra.run.smithi049.stdout:5202 write 0xb17ebf7 thru 0xb184d08 (0x6112 bytes) 2024-04-24T18:08:55.213 INFO:teuthology.orchestra.run.smithi049.stdout:5203 read 0x4e291bf thru 0x4e2db4a (0x498c bytes) 2024-04-24T18:08:55.213 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:55.213 INFO:teuthology.orchestra.run.smithi049.stdout:5205 writesame 0x5994f61 thru 0x59a1c6d data_size 0x1(0xcd0d bytes) 2024-04-24T18:08:55.226 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:55.233+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:55.230 INFO:teuthology.orchestra.run.smithi049.stdout:5206 write 0x2d3abb5 thru 0x2d456de (0xab2a bytes) 2024-04-24T18:08:55.278 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:55.278 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:55.279 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:55.279 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:55.279 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:55.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:08:55.318 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:55.322 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:55.322 INFO:teuthology.orchestra.run.smithi049.stdout:5209 read 0x30a20df thru 0x30a892a (0x684c bytes) 2024-04-24T18:08:55.322 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:55.322 INFO:teuthology.orchestra.run.smithi049.stdout:5211 writesame 0xa3bc9ee thru 0xa3c0e83 data_size 0x1(0x4496 bytes) 2024-04-24T18:08:55.326 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:55.334+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:55.330 INFO:teuthology.orchestra.run.smithi049.stdout:5212 write 0x4a6a267 thru 0x4a6d9e2 (0x377c bytes) 2024-04-24T18:08:55.332 INFO:teuthology.orchestra.run.smithi049.stdout:5213 read 0x6a0535 thru 0x6acdea (0xc8b6 bytes) 2024-04-24T18:08:55.337 INFO:teuthology.orchestra.run.smithi049.stdout:5214 trunc from 0xc1f429d to 0x82a6a59 2024-04-24T18:08:55.340 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:55.376 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:55.411 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:55.426 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:55.435+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:08:55.431 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:55.450 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:55.455 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:55.455 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:55.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:55.455 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:109 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.843229076, 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080175339999999998, 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 109 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616226+0000", 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616225+0000", 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616268+0000", 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3466e-05 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616278+0000", 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2229999999999996e-06 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616283+0000", 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0300000000000001e-06 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619090+0000", 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0028071870000000001 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619141+0000", 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0636000000000001e-05 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619219+0000", 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8782999999999994e-05 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621160+0000", 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019411090000000001 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624149+0000", 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0029884109999999998 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624196+0000", 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6878000000000001e-05 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624243+0000", 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7499e-05 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.843160023999999, 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.461 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:08:55.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:08:55.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.843135609000001, 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:55.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:08:55.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:08:55.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:08:55.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:08:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.84311246, 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:08:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:08:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:08:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:08:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:08:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.843031891999999, 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:08:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:08:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:08:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:08:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.843007082, 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:08:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.842986969999998, 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.842916306999996, 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:08:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.842904533999999, 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:08:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.422582704999996, 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.422535522999993, 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.422510402, 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:08:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:08:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.422458268, 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 94.422394804000007, 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:08:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:08:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:08:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:157 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.911189132000004, 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074242120000000003, 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 157 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548266+0000", 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548265+0000", 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548270+0000", 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9869999999999999e-06 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548276+0000", 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0789999999999997e-06 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548286+0000", 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4420000000000004e-06 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550001+0000", 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017154900000000001 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550050+0000", 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8795000000000003e-05 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550125+0000", 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5073000000000004e-05 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551277+0000", 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0011520440000000001 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555608+0000", 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0043311319999999997 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555649+0000", 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0942000000000002e-05 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555690+0000", 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0760000000000003e-05 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:158 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.911164104999997, 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007457389, 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 158 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548291+0000", 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548290+0000", 2024-04-24T18:08:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548342+0000", 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2135000000000001e-05 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548358+0000", 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5440000000000001e-05 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548363+0000", 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.792e-06 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550411+0000", 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020484259999999999 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550455+0000", 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3894999999999999e-05 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550530+0000", 2024-04-24T18:08:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4606999999999999e-05 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551699+0000", 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001169565 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555698+0000", 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039988360000000004 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555721+0000", 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-05 2024-04-24T18:08:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555748+0000", 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7291e-05 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:159 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e17)", 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.911084868000003, 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0074382320000000004, 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 159 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548370+0000", 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548369+0000", 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548374+0000", 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3930000000000001e-06 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548380+0000", 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6100000000000002e-06 2024-04-24T18:08:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.548387+0000", 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9840000000000004e-06 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550922+0000", 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0025347849999999999 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.550972+0000", 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0170999999999998e-05 2024-04-24T18:08:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551048+0000", 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6088999999999996e-05 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.551920+0000", 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000871425 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555755+0000", 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0038353559999999998 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555787+0000", 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2345000000000003e-05 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.555808+0000", 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0625999999999999e-05 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 62.648896086999997, 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 62.647790018000002, 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:08:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.4926183, 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:08:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:55.558 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:55.560 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:55.560 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:55.561 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:08:55.561 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:55.568+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:55.561 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:55.562 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:55.613 INFO:teuthology.orchestra.run.smithi049.stdout:5216 read 0x73e5bd9 thru 0x73f0ca3 (0xb0cb bytes) 2024-04-24T18:08:55.613 INFO:teuthology.orchestra.run.smithi049.stdout:5217 read 0x16cb14a thru 0x16d6f3a (0xbdf1 bytes) 2024-04-24T18:08:55.613 INFO:teuthology.orchestra.run.smithi049.stdout:5218 punch from 0x5de575c to 0x5df281b, (0xd0bf bytes) 2024-04-24T18:08:55.613 INFO:teuthology.orchestra.run.smithi049.stdout:5219 read 0x1c501ec thru 0x1c589b7 (0x87cc bytes) 2024-04-24T18:08:55.614 INFO:teuthology.orchestra.run.smithi049.stdout:5223 read 0x5a7ded thru 0x5aae7f (0x3093 bytes) 2024-04-24T18:08:55.624 INFO:teuthology.orchestra.run.smithi049.stdout:5224 writesame 0xade5ef0 thru 0xadf2e42 data_size 0x37(0xcf53 bytes) 2024-04-24T18:08:55.631 INFO:teuthology.orchestra.run.smithi049.stdout:5225 writesame 0xb61259a thru 0xb6181be data_size 0x3(0x5c25 bytes) 2024-04-24T18:08:55.631 INFO:teuthology.orchestra.run.smithi049.stdout:5227 punch from 0x38b2736 to 0x38ba4bc, (0x7d86 bytes) 2024-04-24T18:08:55.640 INFO:teuthology.orchestra.run.smithi049.stdout:5228 writesame 0xe0876bd thru 0xe095df8 data_size 0x6(0xe73c bytes) 2024-04-24T18:08:55.640 INFO:teuthology.orchestra.run.smithi049.stdout:5229 punch from 0x4034b07 to 0x4034dfe, (0x2f7 bytes) 2024-04-24T18:08:55.650 INFO:teuthology.orchestra.run.smithi049.stdout:5230 trunc from 0xe095df9 to 0x4f2a45a 2024-04-24T18:08:55.662 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:55.670+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:55.691 INFO:teuthology.orchestra.run.smithi049.stdout:5233 trunc from 0x4f2a45a to 0xb57339 2024-04-24T18:08:55.763 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:55.771+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:55.782 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:55.782 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:55.792 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:08:55.877 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:55.885+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:55.900 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.4 to 1 (10000) 2024-04-24T18:08:55.916 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-24T18:08:56.000 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:56.008+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:56.015 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:56.015 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:56.026 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:08:56.071 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:56.076+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:56.166 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:56.173+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:56.206 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:56.206 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:56.216 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:08:56.267 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:56.274+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:56.344 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:56.344 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:56.354 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:08:56.395 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:56.403+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:56.468 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:56.476+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:56.522 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:56.522 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:56.532 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:08:56.575 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:56.583+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:56.660 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:56.660 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:56.670 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:56.677+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:56.671 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:08:56.771 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:56.779+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:56.840 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:56.840 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:56.850 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:08:56.891 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.5 to 1 (10000) 2024-04-24T18:08:56.891 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:56.898+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:56.904 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-24T18:08:56.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:08:56.992 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:57.000+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:57.038 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:57.038 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:57.049 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:08:57.090 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:57.098+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:57.175 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:57.183+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:57.221 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:57.221 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:57.231 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:08:57.275 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:57.284+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:57.376 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:57.384+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:57.402 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:57.402 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:57.412 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:08:57.477 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:57.485+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:08:57.563 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:08:57.578 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:57.586+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:57.581 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:08:57.581 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:08:57.592 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:08:57.678 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:57.686+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:57.710 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:08:57.729 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:08:57.748 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:08:57.765 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:57.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635764184999999, 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.766 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:08:57.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:08:57.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635749644999997, 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.770 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:08:57.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635734794000001, 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:08:57.774 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:57.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:08:57.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635719645000002, 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.778 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:08:57.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:08:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:08:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:08:57.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635704859000001, 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:08:57.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:08:57.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:57.787 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635690269999998, 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635667445000003, 2024-04-24T18:08:57.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:08:57.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:08:57.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635651791000001, 2024-04-24T18:08:57.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.800 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635636669, 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:57.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.803 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635620852999999, 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:08:57.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:08:57.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635605851000001, 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635590987, 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.813 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:57.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:08:57.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:08:57.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635569869999998, 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:08:57.817 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.821 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635554640000002, 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.635539903000002, 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.630438693000002, 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:08:57.833 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.630402846999999, 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.630382840000003, 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.838 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.628561648999998, 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.842 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:08:57.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:57.846 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 45.628536369000003, 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:57.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:08:57.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:08:57.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:57.851 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:57.851 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:57.852 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:08:57.852 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:08:57.853 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:57.860+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:57.895 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.6 to 1 (10000) 2024-04-24T18:08:57.895 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:08:57.914 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-24T18:08:57.918 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:08:57.944 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:08:57.953 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:08:57.961+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:08:57.967 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:08:57.989 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:08:57.997 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:08:57.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:08:57.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:08:57.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:08:57.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:08:57.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:08:57.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:08:57.998 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:08:58.011 INFO:teuthology.orchestra.run.smithi049.stdout:5235 trunc from 0xb57339 to 0xd369672 2024-04-24T18:08:58.017 INFO:teuthology.orchestra.run.smithi049.stdout:5238 write 0x6e65971 thru 0x6e72331 (0xc9c1 bytes) 2024-04-24T18:08:58.031 INFO:teuthology.orchestra.run.smithi049.stdout:5239 trunc from 0xd369672 to 0x5ea448b 2024-04-24T18:08:58.042 INFO:teuthology.orchestra.run.smithi049.stdout:5240 punch from 0x11fd762 to 0x12027d3, (0x5071 bytes) 2024-04-24T18:08:58.047 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:58.047 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:08:58.047 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:58.053 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:58.061+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:58.056 INFO:teuthology.orchestra.run.smithi049.stdout:5243 writesame 0xc849d36 thru 0xc84ab20 data_size 0x1(0xdeb bytes) 2024-04-24T18:08:58.056 INFO:teuthology.orchestra.run.smithi049.stdout:5247 writesame 0x434b661 thru 0x4359ad7 data_size 0xd(0xe477 bytes) 2024-04-24T18:08:58.062 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:08:58.062 INFO:teuthology.orchestra.run.smithi049.stdout:5248 write 0x3853286 thru 0x38580a0 (0x4e1b bytes) 2024-04-24T18:08:58.064 INFO:teuthology.orchestra.run.smithi049.stdout:5250 punch from 0x3616b55 to 0x361a206, (0x36b1 bytes) 2024-04-24T18:08:58.066 INFO:teuthology.orchestra.run.smithi049.stdout:5251 trunc from 0xc84ab21 to 0xb87acf0 2024-04-24T18:08:58.070 INFO:teuthology.orchestra.run.smithi049.stdout:5252 writesame 0x34d17f2 thru 0x34dce18 data_size 0x11(0xb627 bytes) 2024-04-24T18:08:58.075 INFO:teuthology.orchestra.run.smithi049.stdout:5253 punch from 0x4f9c719 to 0x4fa7386, (0xac6d bytes) 2024-04-24T18:08:58.075 INFO:teuthology.orchestra.run.smithi049.stdout:5255 trunc from 0xb87acf0 to 0xb5b2487 2024-04-24T18:08:58.081 INFO:teuthology.orchestra.run.smithi049.stdout:5258 writesame 0x991f0ac thru 0x9926e6a data_size 0x1(0x7dbf bytes) 2024-04-24T18:08:58.102 INFO:teuthology.orchestra.run.smithi049.stdout:5261 read 0xab1744a thru 0xab26a60 (0xf617 bytes) 2024-04-24T18:08:58.102 INFO:teuthology.orchestra.run.smithi049.stdout:5262 write 0xa8f09fe thru 0xa8ff8b8 (0xeebb bytes) 2024-04-24T18:08:58.106 INFO:teuthology.orchestra.run.smithi049.stdout:5263 write 0x4ef91d2 thru 0x4f02ffe (0x9e2d bytes) 2024-04-24T18:08:58.111 INFO:teuthology.orchestra.run.smithi049.stdout:5264 write 0x9791711 thru 0x979ec97 (0xd587 bytes) 2024-04-24T18:08:58.115 INFO:teuthology.orchestra.run.smithi049.stdout:5265 read 0x63ab538 thru 0x63b8995 (0xd45e bytes) 2024-04-24T18:08:58.116 INFO:teuthology.orchestra.run.smithi049.stdout:5266 punch from 0x9d85840 to 0x9d880a4, (0x2864 bytes) 2024-04-24T18:08:58.116 INFO:teuthology.orchestra.run.smithi049.stdout:5267 punch from 0x4961bbf to 0x496aadb, (0x8f1c bytes) 2024-04-24T18:08:58.116 INFO:teuthology.orchestra.run.smithi049.stdout:5268 punch from 0x8fcd917 to 0x8fd413a, (0x6823 bytes) 2024-04-24T18:08:58.116 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:58.116 INFO:teuthology.orchestra.run.smithi049.stdout:5272 writesame 0x9b629ce thru 0x9b6ae86 data_size 0x1(0x84b9 bytes) 2024-04-24T18:08:58.139 INFO:teuthology.orchestra.run.smithi049.stdout:5273 write 0xd61391d thru 0xd615856 (0x1f3a bytes) 2024-04-24T18:08:58.142 INFO:teuthology.orchestra.run.smithi049.stdout:5275 trunc from 0xd615857 to 0xd5621ac 2024-04-24T18:08:58.148 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:58.148 INFO:teuthology.orchestra.run.smithi049.stdout:5277 punch from 0x2f48258 to 0x2f51c77, (0x9a1f bytes) 2024-04-24T18:08:58.148 INFO:teuthology.orchestra.run.smithi049.stdout:5278 write 0x5cde509 thru 0x5ce3540 (0x5038 bytes) 2024-04-24T18:08:58.152 INFO:teuthology.orchestra.run.smithi049.stdout:5279 write 0x90333e0 thru 0x9039038 (0x5c59 bytes) 2024-04-24T18:08:58.154 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:58.162+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:58.170 INFO:teuthology.orchestra.run.smithi049.stdout:5280 trunc from 0xd5621ac to 0x5d4090 2024-04-24T18:08:58.194 INFO:teuthology.orchestra.run.smithi049.stdout:5281 write 0x26848ce thru 0x269447a (0xfbad bytes) 2024-04-24T18:08:58.196 INFO:teuthology.orchestra.run.smithi049.stdout:5282 punch from 0x19a7955 to 0x19a9f14, (0x25bf bytes) 2024-04-24T18:08:58.196 INFO:teuthology.orchestra.run.smithi049.stdout:5283 trunc from 0x269447b to 0x26c8bf3 2024-04-24T18:08:58.214 INFO:teuthology.orchestra.run.smithi049.stdout:5284 trunc from 0x26c8bf3 to 0xe56c9cf 2024-04-24T18:08:58.215 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:58.216 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:08:58.216 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:08:58.216 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:08:58.216 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:58.219 INFO:teuthology.orchestra.run.smithi049.stdout:5285 writesame 0x540eafb thru 0x541b0bd data_size 0x1(0xc5c3 bytes) 2024-04-24T18:08:58.227 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:08:58.244 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:58.244 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:58.244 INFO:teuthology.orchestra.run.smithi049.stdout:5290 writesame 0x14103e2 thru 0x141e0c6 data_size 0x1(0xdce5 bytes) 2024-04-24T18:08:58.255 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:08:58.264+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:08:58.260 INFO:teuthology.orchestra.run.smithi049.stdout:5292 write 0x7eb428e thru 0x7ebe14a (0x9ebd bytes) 2024-04-24T18:08:58.265 INFO:teuthology.orchestra.run.smithi049.stdout:5293 read 0x60f5b5a thru 0x60fc25f (0x6706 bytes) 2024-04-24T18:08:58.271 INFO:teuthology.orchestra.run.smithi049.stdout:5294 write 0xedb17b4 thru 0xedb6297 (0x4ae4 bytes) 2024-04-24T18:08:58.276 INFO:teuthology.orchestra.run.smithi049.stdout:5295 write 0x7838e9d thru 0x78473c2 (0xe526 bytes) 2024-04-24T18:08:58.279 INFO:teuthology.orchestra.run.smithi049.stdout:5296 write 0x16c4301 thru 0x16cb607 (0x7307 bytes) 2024-04-24T18:08:58.287 INFO:teuthology.orchestra.run.smithi049.stdout:5297 trunc from 0xedb6298 to 0x9f2ace9 2024-04-24T18:08:58.297 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:58.297 INFO:teuthology.orchestra.run.smithi049.stdout:5300 punch from 0x45bfcad to 0x45cc65a, (0xc9ad bytes) 2024-04-24T18:08:58.297 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:58.304 INFO:teuthology.orchestra.run.smithi049.stdout:5302 write 0xa4a7a2d thru 0xa4af249 (0x781d bytes) 2024-04-24T18:08:58.308 INFO:teuthology.orchestra.run.smithi049.stdout:5304 read 0x4badc37 thru 0x4bb2cd0 (0x509a bytes) 2024-04-24T18:08:58.308 INFO:teuthology.orchestra.run.smithi049.stdout:5305 read 0x789a12b thru 0x78a6955 (0xc82b bytes) 2024-04-24T18:08:58.309 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:58.309 INFO:teuthology.orchestra.run.smithi049.stdout:5307 punch from 0x7e58e55 to 0x7e5e921, (0x5acc bytes) 2024-04-24T18:08:58.322 INFO:teuthology.orchestra.run.smithi049.stdout:5309 write 0xed00101 thru 0xed0f3c6 (0xf2c6 bytes) 2024-04-24T18:08:58.323 INFO:teuthology.orchestra.run.smithi049.stdout:5310 clone 49 order 20 su 4096 sc 11 2024-04-24T18:08:58.356 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:58.364+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:58.387 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.216621918000001, 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:08:58.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.196982792, 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.392 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:08:58.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.395 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.196951069999997, 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.399 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.196927653000003, 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.403 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.196897579999998, 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.404 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:08:58.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:58.408 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195852432999999, 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:08:58.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:08:58.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195835201999998, 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:08:58.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.416 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195818203999998, 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:08:58.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:08:58.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.420 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195799522999998, 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:58.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:08:58.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:08:58.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.424 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195782055000002, 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195765930999997, 2024-04-24T18:08:58.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:58.432 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195734825000002, 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:08:58.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195715624000002, 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.440 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.195694582000002, 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:08:58.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:08:58.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:08:58.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:08:58.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.194939634000001, 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.194913132000003, 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.449 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.194895602000003, 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:08:58.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.193432442000002, 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.181593794000001, 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 46.181577451000003, 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:58.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:08:58.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:08:58.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:08:58.468 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:08:58.469 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:08:58.470 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:08:58.471 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:58.478+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:58.571 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:08:58.579+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:08:58.594 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:58.594 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:58.594 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:58.594 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:58.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:08:58.672 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:08:58.680+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:08:58.766 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:58.766 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:58.766 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:58.766 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:58.766 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:58.774 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:08:58.782+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:08:58.776 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:08:58.875 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:58.883+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:58.900 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.7 to 1 (10000) 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.569725152000004, 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:08:58.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.569709246000002, 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.906 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.569657278999998, 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.910 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:08:58.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:08:58.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.568733804999994, 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:08:58.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:08:58.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.568717172999996, 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:08:58.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.56801557, 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.567999068000006, 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:08:58.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:08:58.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.567096202000002, 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.931 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:08:58.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.567088177000002, 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.938 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:08:58.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:08:58.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.566024034999998, 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.566008065999995, 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:08:58.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:08:58.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.505809626999998, 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:08:58.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:08:58.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.956 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136785 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 215040~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.336800538999999, 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430404999999997, 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136785 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571951+0000", 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571953+0000", 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.08e-06 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571959+0000", 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3469999999999998e-06 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571962+0000", 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8079999999999998e-06 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614268+0000", 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042306032 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614315+0000", 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6529999999999997e-05 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614382+0000", 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6915000000000006e-05 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.336785594999998, 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.964 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.336769361000002, 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:08:58.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.336754141, 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.336726561999999, 2024-04-24T18:08:58.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:58.977 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.336708260999998, 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:08:58.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:08:58.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.336691962, 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:08:58.985 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 5.028289343, 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:58.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:58.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:58.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:58.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.991 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:58.991 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:58.991 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:58.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-24T18:08:58.993 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone48 from 0xed0f3c7 (overlap 0x5d4090) to 0x0 2024-04-24T18:08:58.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:08:58.994 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:59.000+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:59.032 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #47, image image_client.0-clone47 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent48 2024-04-24T18:08:59.095 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:08:59.103+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:08:59.196 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:59.202+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:189680 4.15 4.e341f555 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.238905+0000", 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.000964715, 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00099180100000000005, 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 189680 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238905+0000", 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238905+0000", 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238905+0000", 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238907+0000", 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.447e-06 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238914+0000", 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9750000000000001e-06 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238919+0000", 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.442e-06 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:189681 4.15 4.e341f555 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.238923+0000", 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00094693999999999996, 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001062527, 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 189681 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238923+0000", 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238923+0000", 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238923+0000", 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238925+0000", 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3489999999999999e-06 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238931+0000", 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7599999999999999e-06 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238934+0000", 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5429999999999998e-06 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.238 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:189682 4.15 4.e341f555 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.238938+0000", 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00093162699999999998, 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001111138, 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 189682 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238938+0000", 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238938+0000", 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238938+0000", 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238940+0000", 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0949999999999998e-06 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238946+0000", 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5149999999999997e-06 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238950+0000", 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8890000000000004e-06 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:189676 4.15 4:aaaf82c7:::rbd_data.11747889028a.0000000000000024:32 [read 167936~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.238836+0000", 2024-04-24T18:08:59.242 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00103429, 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001263686, 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "reached pg", 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 189676 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238836+0000", 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238836+0000", 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238835+0000", 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238838+0000", 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1830000000000001e-06 2024-04-24T18:08:59.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238843+0000", 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8599999999999998e-06 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238848+0000", 2024-04-24T18:08:59.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.532e-06 2024-04-24T18:08:59.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.240026+0000", 2024-04-24T18:08:59.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001177853 2024-04-24T18:08:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.247 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:189677 4.15 4.e341f555 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:08:59.248 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.238852+0000", 2024-04-24T18:08:59.248 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.0010178889999999999, 2024-04-24T18:08:59.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001285429, 2024-04-24T18:08:59.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:59.248 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.249 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.249 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.249 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 189677 2024-04-24T18:08:59.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.249 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238852+0000", 2024-04-24T18:08:59.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238852+0000", 2024-04-24T18:08:59.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238852+0000", 2024-04-24T18:08:59.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238854+0000", 2024-04-24T18:08:59.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1119999999999999e-06 2024-04-24T18:08:59.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238859+0000", 2024-04-24T18:08:59.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3789999999999997e-06 2024-04-24T18:08:59.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238863+0000", 2024-04-24T18:08:59.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4470000000000002e-06 2024-04-24T18:08:59.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.255 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.256 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:189678 4.15 4.e341f555 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:08:59.256 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.238867+0000", 2024-04-24T18:08:59.256 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.001003159, 2024-04-24T18:08:59.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013035709999999999, 2024-04-24T18:08:59.256 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.257 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:59.257 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.257 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.257 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 189678 2024-04-24T18:08:59.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.258 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238867+0000", 2024-04-24T18:08:59.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238867+0000", 2024-04-24T18:08:59.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238866+0000", 2024-04-24T18:08:59.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238869+0000", 2024-04-24T18:08:59.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2699999999999999e-06 2024-04-24T18:08:59.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238874+0000", 2024-04-24T18:08:59.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7030000000000003e-06 2024-04-24T18:08:59.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238878+0000", 2024-04-24T18:08:59.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4800000000000001e-06 2024-04-24T18:08:59.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.263 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.264 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.264 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:189679 4.15 4.e341f555 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:08:59.264 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.238882+0000", 2024-04-24T18:08:59.264 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.00098764699999999992, 2024-04-24T18:08:59.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001318545, 2024-04-24T18:08:59.265 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "queued for pg", 2024-04-24T18:08:59.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 189679 2024-04-24T18:08:59.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.266 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238882+0000", 2024-04-24T18:08:59.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238882+0000", 2024-04-24T18:08:59.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238882+0000", 2024-04-24T18:08:59.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238884+0000", 2024-04-24T18:08:59.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0499999999999999e-06 2024-04-24T18:08:59.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238890+0000", 2024-04-24T18:08:59.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4820000000000002e-06 2024-04-24T18:08:59.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.238893+0000", 2024-04-24T18:08:59.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.517e-06 2024-04-24T18:08:59.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.271 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.272 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.272 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.272 INFO:teuthology.orchestra.run.smithi078.stdout: ], 2024-04-24T18:08:59.272 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 7 2024-04-24T18:08:59.272 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:59.274 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:08:59.316 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:59.324+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:59.395 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:59.404+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:59.414 INFO:teuthology.orchestra.run.smithi049.stdout:5312 write 0x1a2ec4b thru 0x1a38b6d (0x9f23 bytes) 2024-04-24T18:08:59.459 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:59.459 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:59.459 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:08:59.459 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:08:59.459 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:59.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:08:59.496 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:59.504+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:59.555 INFO:teuthology.orchestra.run.smithi049.stdout:5315 write 0x1bf6432 thru 0x1bf8949 (0x2518 bytes) 2024-04-24T18:08:59.558 INFO:teuthology.orchestra.run.smithi049.stdout:5316 writesame 0x5eb1ce6 thru 0x5eb3d70 data_size 0x1(0x208b bytes) 2024-04-24T18:08:59.579 INFO:teuthology.orchestra.run.smithi049.stdout:5317 write 0x7e98391 thru 0x7ea0541 (0x81b1 bytes) 2024-04-24T18:08:59.597 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:59.605+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:59.615 INFO:teuthology.orchestra.run.smithi049.stdout:5318 read 0xcd62428 thru 0xcd6e820 (0xc3f9 bytes) 2024-04-24T18:08:59.616 INFO:teuthology.orchestra.run.smithi049.stdout:5319 punch from 0xe0ed51 to 0xe151b6, (0x6465 bytes) 2024-04-24T18:08:59.616 INFO:teuthology.orchestra.run.smithi049.stdout:5320 read 0x577e490 thru 0x5780031 (0x1ba2 bytes) 2024-04-24T18:08:59.616 INFO:teuthology.orchestra.run.smithi049.stdout:5321 write 0xaa375ad thru 0xaa44d2b (0xd77f bytes) 2024-04-24T18:08:59.659 INFO:teuthology.orchestra.run.smithi049.stdout:5322 write 0x3acd927 thru 0x3ad432e (0x6a08 bytes) 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.324735341, 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.324727211999999, 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:08:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.324719189, 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.324710276000001, 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:08:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.324701457, 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:08:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:08:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:08:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:08:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.324692325000001, 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:08:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:08:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323872066, 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:08:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:08:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323844866999998, 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:08:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323827396999999, 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323812057000001, 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:08:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323787978999999, 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:08:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:08:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:08:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:08:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:08:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:08:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:08:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323772414, 2024-04-24T18:08:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:08:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:08:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:08:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:08:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:08:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:08:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:08:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:08:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:08:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:08:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:08:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:08:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:08:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:08:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:08:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:08:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323757452999999, 2024-04-24T18:08:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:08:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:08:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:08:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:08:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:08:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:08:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.746 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.746 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:08:59.746 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:08:59.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:08:59.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:08:59.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:08:59.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:08:59.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:08:59.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:08:59.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:08:59.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:08:59.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.750 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.750 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.750 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:59.751 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323740514000001, 2024-04-24T18:08:59.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:08:59.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.751 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.751 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.752 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.752 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:08:59.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.752 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:59.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:59.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:08:59.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:08:59.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:08:59.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:08:59.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:08:59.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:08:59.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:08:59.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:08:59.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:08:59.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:08:59.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:08:59.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:08:59.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:08:59.760 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.760 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.760 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.761 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.761 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:59.761 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323724109, 2024-04-24T18:08:59.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:08:59.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.762 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.762 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.762 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.762 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:08:59.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.763 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:59.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:08:59.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:08:59.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:08:59.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:08:59.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.766 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.766 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.766 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:08:59.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:08:59.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.767 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:08:59.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:08:59.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:08:59.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:08:59.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:08:59.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:08:59.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:08:59.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:08:59.770 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.770 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.770 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.771 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.771 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:59.771 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323707670000001, 2024-04-24T18:08:59.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:08:59.772 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.772 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.772 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.772 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.772 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:08:59.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.773 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.773 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.773 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:59.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:59.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.775 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.775 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:08:59.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.776 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:08:59.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:08:59.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:08:59.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:08:59.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:08:59.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:08:59.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:08:59.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:08:59.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:08:59.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:08:59.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.781 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.781 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.781 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:59.782 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323692829999999, 2024-04-24T18:08:59.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:08:59.782 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.782 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.782 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.782 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.783 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.783 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:08:59.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.783 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:59.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:59.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:08:59.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:08:59.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:08:59.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:08:59.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323676406000001, 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323659191000001, 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 29.323641726000002, 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:08:59.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:08:59.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:08:59.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.800 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:08:59.801 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:08:59.801 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:59.803 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:08:59.803 INFO:teuthology.orchestra.run.smithi049.stdout:5328 punch from 0xe1ab265 to 0xe1aef80, (0x3d1b bytes) 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5329 write 0x3a2da7e thru 0x3a392be (0xb841 bytes) 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5331 read 0x6f3627c thru 0x6f455d2 (0xf357 bytes) 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5334 trunc from 0xed0f3c7 to 0x8eb5818 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5335 read 0x4f7fb58 thru 0x4f80dce (0x1277 bytes) 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5337 read 0x4531001 thru 0x4538cb7 (0x7cb7 bytes) 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5341 read 0x56b159a thru 0x56ba381 (0x8de8 bytes) 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5342 writesame 0x9b8e94c thru 0x9b97297 data_size 0x74(0x894c bytes) 2024-04-24T18:08:59.804 INFO:teuthology.orchestra.run.smithi049.stdout:5343 write 0x585c303 thru 0x586b058 (0xed56 bytes) 2024-04-24T18:08:59.804 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:08:59.810+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:08:59.843 INFO:teuthology.orchestra.run.smithi049.stdout:5346 write 0x83d73e2 thru 0x83df79d (0x83bc bytes) 2024-04-24T18:08:59.850 INFO:teuthology.orchestra.run.smithi049.stdout:5347 read 0x381357a thru 0x381c045 (0x8acc bytes) 2024-04-24T18:08:59.851 INFO:teuthology.orchestra.run.smithi049.stdout:5348 writesame 0x584f997 thru 0x585c2b8 data_size 0xa(0xc922 bytes) 2024-04-24T18:08:59.862 INFO:teuthology.orchestra.run.smithi049.stdout:5349 trunc from 0x9b97298 to 0x4663d4f 2024-04-24T18:08:59.903 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:08:59.911+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:08:59.906 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.1 to 1 (10000) 2024-04-24T18:08:59.915 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:08:59.922 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-24T18:08:59.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:08:59.951 INFO:teuthology.orchestra.run.smithi049.stdout:5350 read 0x2db3ad9 thru 0x2db91e3 (0x570b bytes) 2024-04-24T18:08:59.952 INFO:teuthology.orchestra.run.smithi049.stdout:5351 punch from 0x4303eb2 to 0x430a6fa, (0x6848 bytes) 2024-04-24T18:08:59.952 INFO:teuthology.orchestra.run.smithi049.stdout:5353 read 0x41079c6 thru 0x410f335 (0x7970 bytes) 2024-04-24T18:08:59.953 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.960 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:08:59.960 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:08:59.960 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:08:59.960 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:08:59.963 INFO:teuthology.orchestra.run.smithi049.stdout:5356 trunc from 0x4663d4f to 0xd80324e 2024-04-24T18:08:59.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:08:59.972 INFO:teuthology.orchestra.run.smithi049.stdout:5358 trunc from 0xd80324e to 0xaef7972 2024-04-24T18:08:59.978 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.978 INFO:teuthology.orchestra.run.smithi049.stdout:5361 writesame 0x974e9f0 thru 0x9754056 data_size 0x3(0x5667 bytes) 2024-04-24T18:08:59.988 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.988 INFO:teuthology.orchestra.run.smithi049.stdout:5363 punch from 0x2270f to 0x2a442, (0x7d33 bytes) 2024-04-24T18:08:59.989 INFO:teuthology.orchestra.run.smithi049.stdout:5364 read 0x9c85c19 thru 0x9c85e21 (0x209 bytes) 2024-04-24T18:08:59.993 INFO:teuthology.orchestra.run.smithi049.stdout:5366 writesame 0xaf54cb2 thru 0xaf622ed data_size 0x2(0xd63c bytes) 2024-04-24T18:08:59.994 INFO:teuthology.orchestra.run.smithi049.stdout:5368 read 0x6688e5a thru 0x6698886 (0xfa2d bytes) 2024-04-24T18:08:59.994 INFO:teuthology.orchestra.run.smithi049.stdout:5369 read 0x4c08abf thru 0x4c14416 (0xb958 bytes) 2024-04-24T18:08:59.994 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.994 INFO:teuthology.orchestra.run.smithi049.stdout:5371 read 0x8a770fd thru 0x8a83622 (0xc526 bytes) 2024-04-24T18:08:59.994 INFO:teuthology.orchestra.run.smithi049.stdout:5372 read 0x18f6dcb thru 0x1900d8b (0x9fc1 bytes) 2024-04-24T18:08:59.995 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.995 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.995 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:08:59.995 INFO:teuthology.orchestra.run.smithi049.stdout:5377 writesame 0x25c621e thru 0x25d4858 data_size 0x1(0xe63b bytes) 2024-04-24T18:09:00.011 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:00.014+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:00.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:00.093 INFO:teuthology.orchestra.run.smithi049.stdout:5378 write 0x6f7ea7d thru 0x6f827ca (0x3d4e bytes) 2024-04-24T18:09:00.104 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:00.113+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:00.105 INFO:teuthology.orchestra.run.smithi049.stdout:5379 flatten 2024-04-24T18:09:00.206 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:00.214+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:00.220 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:00.220 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:00.237 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:00.317 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:00.315+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:00.321 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:00.402 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:00.411 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:00.418+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:00.431 INFO:teuthology.orchestra.run.smithi049.stdout:5381 write 0x1b150e2 thru 0x1b17e3d (0x2d5c bytes) 2024-04-24T18:09:00.438 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:00.442 INFO:teuthology.orchestra.run.smithi049.stdout:5382 write 0xc51706d thru 0xc523f95 (0xcf29 bytes) 2024-04-24T18:09:00.461 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.852234654, 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.462 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:00.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:00.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.852210239000001, 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:00.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:00.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:00.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:00.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.472 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.852187090000001, 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:00.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:00.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:00.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:00.477 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.852106522, 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:00.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:00.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:00.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.852081712, 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.483 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:00.488 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.852061599999999, 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.493 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.851990936999997, 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:00.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:00.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:00.498 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.851979163999999, 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:00.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:00.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:00.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.431657334999997, 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:00.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:00.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:00.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:00.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.431610152999994, 2024-04-24T18:09:00.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.431585032000001, 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:00.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:00.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.431532898, 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:00.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:00.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:00.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:00.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 99.431469433999993, 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:00.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:00.530 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 67.657970716999998, 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 67.656864647999996, 2024-04-24T18:09:00.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:00.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 28.501692930000001, 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:00.538 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:00.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:00.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.439738986, 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.542 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.43845190299999998, 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.099275831999999994, 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:00.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:00.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.553 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 0.099186137999999993, 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.557 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.558 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:00.558 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:00.558 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:00.559 INFO:teuthology.orchestra.run.smithi049.stdout:5383 read 0x39cf4a3 thru 0x39d4bfe (0x575c bytes) 2024-04-24T18:09:00.559 INFO:teuthology.orchestra.run.smithi049.stdout:5384 read 0xaa5ca16 thru 0xaa6b79f (0xed8a bytes) 2024-04-24T18:09:00.559 INFO:teuthology.orchestra.run.smithi049.stdout:5385 writesame 0x2589cfa thru 0x25987b7 data_size 0x1(0xeabe bytes) 2024-04-24T18:09:00.559 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:00.560 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:5386 writesame 0x3a237ae thru 0x3a23db3 data_size 0x3(0x606 bytes) 2024-04-24T18:09:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:5387 read 0x14a3173 thru 0x14a58ba (0x2748 bytes) 2024-04-24T18:09:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:5388 punch from 0xa7f3e4b to 0xa7f9572, (0x5727 bytes) 2024-04-24T18:09:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:00.561 INFO:teuthology.orchestra.run.smithi049.stdout:5389 trunc from 0xc523f96 to 0xf9c7a7 2024-04-24T18:09:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:00.562 INFO:teuthology.orchestra.run.smithi049.stdout:5391 trunc from 0xf9c7a7 to 0x88107b 2024-04-24T18:09:00.564 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:00.565 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:00.565 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:00.565 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:00.567+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:00.566 INFO:teuthology.orchestra.run.smithi049.stdout:5392 read 0x16c67f thru 0x16f852 (0x31d4 bytes) 2024-04-24T18:09:00.566 INFO:teuthology.orchestra.run.smithi049.stdout:5394 read 0x294845 thru 0x2975a0 (0x2d5c bytes) 2024-04-24T18:09:00.574 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":146,"stamp":"2024-04-24T18:08:59.916818+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1329177800,"num_objects":3706,"num_object_clones":673,"num_object_copies":7412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3704,"num_whiteouts":2330,"num_read":227105,"num_read_kb":733141,"num_write":21007,"num_write_kb":473743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18656,"ondisk_log_size":18656,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2543008,"kb_used_data":76840,"kb_used_omap":43,"kb_used_meta":2466004,"kb_avail":752431712,"statfs":{"total":773094113280,"available":770490073088,"internally_reserved":0,"allocated":78684160,"data_stored":662705684,"data_compressed":12970785,"data_compressed_allocated":42500096,"data_compressed_original":634904840,"omap_allocated":44665,"internal_metadata":2525188487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5662667,"num_objects":120,"num_object_clones":4,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-781,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":112,"num_read":10623,"num_read_kb":26606,"num_write":383,"num_write_kb":25315,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":726,"num_bytes_recovered":25845340,"num_keys_recovered":70,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.023764"},"pg_stats":[{"pgid":"4.1f","version":"97'457","reported_seq":5382,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.901318+0000","last_change":"2024-04-24T18:08:55.899667+0000","last_active":"2024-04-24T18:08:55.899445+0000","last_peered":"2024-04-24T18:08:55.671181+0000","last_clean":"2024-04-24T18:08:18.970509+0000","last_became_active":"2024-04-24T18:08:54.974383+0000","last_became_peered":"2024-04-24T18:08:54.974383+0000","last_unstale":"2024-04-24T18:08:55.901318+0000","last_undegraded":"2024-04-24T18:08:55.901318+0000","last_fullsized":"2024-04-24T18:08:55.901318+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:39:05.236590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":48185911,"num_objects":139,"num_object_clones":25,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":94,"num_read":5428,"num_read_kb":27520,"num_write":459,"num_write_kb":11775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"94'1135","reported_seq":5715,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.895044+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:08:55.895044+0000","last_peered":"2024-04-24T18:08:55.895044+0000","last_clean":"2024-04-24T18:08:55.895044+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:08:55.895044+0000","last_undegraded":"2024-04-24T18:08:55.895044+0000","last_fullsized":"2024-04-24T18:08:55.895044+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1135,"log_dups_size":0,"ondisk_log_size":1135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":38392536,"num_objects":139,"num_object_clones":24,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":89,"num_read":9052,"num_read_kb":19486,"num_write":1680,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"99'512","reported_seq":3960,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.935539+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:08:56.935539+0000","last_peered":"2024-04-24T18:08:56.935539+0000","last_clean":"2024-04-24T18:08:56.935539+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:08:56.935539+0000","last_undegraded":"2024-04-24T18:08:56.935539+0000","last_fullsized":"2024-04-24T18:08:56.935539+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":32404686,"num_objects":138,"num_object_clones":25,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":89,"num_read":4444,"num_read_kb":14922,"num_write":490,"num_write_kb":11863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"97'791","reported_seq":7936,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.895918+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:08:55.895918+0000","last_peered":"2024-04-24T18:08:55.895918+0000","last_clean":"2024-04-24T18:08:55.895918+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:08:55.895918+0000","last_undegraded":"2024-04-24T18:08:55.895918+0000","last_fullsized":"2024-04-24T18:08:55.895918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":39279832,"num_objects":108,"num_object_clones":23,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":63,"num_read":9978,"num_read_kb":21879,"num_write":1068,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"97'384","reported_seq":7066,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.927203+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:08:56.927203+0000","last_peered":"2024-04-24T18:08:56.927203+0000","last_clean":"2024-04-24T18:08:56.927203+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:08:56.927203+0000","last_undegraded":"2024-04-24T18:08:56.927203+0000","last_fullsized":"2024-04-24T18:08:56.927203+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":45749737,"num_objects":112,"num_object_clones":24,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":61,"num_read":6776,"num_read_kb":30196,"num_write":366,"num_write_kb":18770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"94'450","reported_seq":10342,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.919230+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:08:56.919230+0000","last_peered":"2024-04-24T18:08:56.919230+0000","last_clean":"2024-04-24T18:08:56.919230+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:08:56.919230+0000","last_undegraded":"2024-04-24T18:08:56.919230+0000","last_fullsized":"2024-04-24T18:08:56.919230+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":41098372,"num_objects":120,"num_object_clones":27,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":68,"num_read":11189,"num_read_kb":27289,"num_write":394,"num_write_kb":11539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"97'545","reported_seq":8037,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900063+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:08:55.900063+0000","last_peered":"2024-04-24T18:08:55.900063+0000","last_clean":"2024-04-24T18:08:55.900063+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:08:55.900063+0000","last_undegraded":"2024-04-24T18:08:55.900063+0000","last_fullsized":"2024-04-24T18:08:55.900063+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":39665781,"num_objects":116,"num_object_clones":19,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":77,"num_read":8848,"num_read_kb":27076,"num_write":565,"num_write_kb":13542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"98'415","reported_seq":8350,"reported_epoch":99,"state":"peering","last_fresh":"2024-04-24T18:08:56.896474+0000","last_change":"2024-04-24T18:08:56.896899+0000","last_active":"2024-04-24T18:08:56.896439+0000","last_peered":"2024-04-24T18:08:56.041651+0000","last_clean":"2024-04-24T18:08:20.733439+0000","last_became_active":"2024-04-24T18:08:54.975461+0000","last_became_peered":"2024-04-24T18:08:54.975461+0000","last_unstale":"2024-04-24T18:08:56.896474+0000","last_undegraded":"2024-04-24T18:08:56.896474+0000","last_fullsized":"2024-04-24T18:08:56.896474+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:02:54.955318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":44496065,"num_objects":103,"num_object_clones":18,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":60,"num_read":8906,"num_read_kb":26427,"num_write":406,"num_write_kb":26815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"97'414","reported_seq":4852,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.002567+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:08:56.002567+0000","last_peered":"2024-04-24T18:08:56.002567+0000","last_clean":"2024-04-24T18:08:56.002567+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:08:56.002567+0000","last_undegraded":"2024-04-24T18:08:56.002567+0000","last_fullsized":"2024-04-24T18:08:56.002567+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":41936818,"num_objects":114,"num_object_clones":26,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":72,"num_read":5026,"num_read_kb":19857,"num_write":381,"num_write_kb":15303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"97'1035","reported_seq":3056,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.008253+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:08:56.008253+0000","last_peered":"2024-04-24T18:08:56.008253+0000","last_clean":"2024-04-24T18:08:56.008253+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:08:56.008253+0000","last_undegraded":"2024-04-24T18:08:56.008253+0000","last_fullsized":"2024-04-24T18:08:56.008253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1035,"log_dups_size":0,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":41063598,"num_objects":122,"num_object_clones":19,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":75,"num_read":6837,"num_read_kb":20698,"num_write":1522,"num_write_kb":17149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-24T18:08:54.963507+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:08:54.963507+0000","last_peered":"2024-04-24T18:08:54.963507+0000","last_clean":"2024-04-24T18:08:54.963507+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:08:54.963507+0000","last_undegraded":"2024-04-24T18:08:54.963507+0000","last_fullsized":"2024-04-24T18:08:54.963507+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-24T18:08:54.963544+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:08:54.963544+0000","last_peered":"2024-04-24T18:08:54.963544+0000","last_clean":"2024-04-24T18:08:54.963544+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:08:54.963544+0000","last_undegraded":"2024-04-24T18:08:54.963544+0000","last_fullsized":"2024-04-24T18:08:54.963544+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"94'499","reported_seq":4408,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.012311+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:08:56.012311+0000","last_peered":"2024-04-24T18:08:56.012311+0000","last_clean":"2024-04-24T18:08:56.012311+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:08:56.012311+0000","last_undegraded":"2024-04-24T18:08:56.012311+0000","last_fullsized":"2024-04-24T18:08:56.012311+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":59713549,"num_objects":122,"num_object_clones":24,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":74,"num_read":4646,"num_read_kb":29075,"num_write":538,"num_write_kb":18441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":88,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898146+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:55.898146+0000","last_peered":"2024-04-24T18:08:55.898146+0000","last_clean":"2024-04-24T18:08:55.898146+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:55.898146+0000","last_undegraded":"2024-04-24T18:08:55.898146+0000","last_fullsized":"2024-04-24T18:08:55.898146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898226+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:55.898226+0000","last_peered":"2024-04-24T18:08:55.898226+0000","last_clean":"2024-04-24T18:08:55.898226+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:55.898226+0000","last_undegraded":"2024-04-24T18:08:55.898226+0000","last_fullsized":"2024-04-24T18:08:55.898226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"99'822","reported_seq":4432,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894848+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:08:56.894848+0000","last_peered":"2024-04-24T18:08:56.894848+0000","last_clean":"2024-04-24T18:08:56.894848+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:08:56.894848+0000","last_undegraded":"2024-04-24T18:08:56.894848+0000","last_fullsized":"2024-04-24T18:08:56.894848+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":29722279,"num_objects":114,"num_object_clones":18,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":82,"num_read":6845,"num_read_kb":12351,"num_write":1025,"num_write_kb":15481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"61'1","reported_seq":89,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900310+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:55.900310+0000","last_peered":"2024-04-24T18:08:55.900310+0000","last_clean":"2024-04-24T18:08:55.900310+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:55.900310+0000","last_undegraded":"2024-04-24T18:08:55.900310+0000","last_fullsized":"2024-04-24T18:08:55.900310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:06.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191347+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:56.191347+0000","last_peered":"2024-04-24T18:08:56.191347+0000","last_clean":"2024-04-24T18:08:56.191347+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:56.191347+0000","last_undegraded":"2024-04-24T18:08:56.191347+0000","last_fullsized":"2024-04-24T18:08:56.191347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"97'733","reported_seq":6276,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.904960+0000","last_change":"2024-04-24T18:08:55.901049+0000","last_active":"2024-04-24T18:08:55.900833+0000","last_peered":"2024-04-24T18:08:55.613013+0000","last_clean":"2024-04-24T18:08:19.722254+0000","last_became_active":"2024-04-24T18:08:54.974371+0000","last_became_peered":"2024-04-24T18:08:54.974371+0000","last_unstale":"2024-04-24T18:08:55.904960+0000","last_undegraded":"2024-04-24T18:08:55.904960+0000","last_fullsized":"2024-04-24T18:08:55.904960+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:09.259454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":53469738,"num_objects":138,"num_object_clones":21,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":88,"num_read":8268,"num_read_kb":36769,"num_write":822,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.893935+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:55.893935+0000","last_peered":"2024-04-24T18:08:55.893935+0000","last_clean":"2024-04-24T18:08:55.893935+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:55.893935+0000","last_undegraded":"2024-04-24T18:08:55.893935+0000","last_fullsized":"2024-04-24T18:08:55.893935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191290+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:56.191290+0000","last_peered":"2024-04-24T18:08:56.191290+0000","last_clean":"2024-04-24T18:08:56.191290+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:56.191290+0000","last_undegraded":"2024-04-24T18:08:56.191290+0000","last_fullsized":"2024-04-24T18:08:56.191290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"94'372","reported_seq":4298,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.992256+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:08:55.992256+0000","last_peered":"2024-04-24T18:08:55.992256+0000","last_clean":"2024-04-24T18:08:55.992256+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:08:55.992256+0000","last_undegraded":"2024-04-24T18:08:55.992256+0000","last_fullsized":"2024-04-24T18:08:55.992256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":30102454,"num_objects":113,"num_object_clones":28,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":66,"num_read":5535,"num_read_kb":23916,"num_write":471,"num_write_kb":12527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'1","reported_seq":89,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.894711+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:55.894711+0000","last_peered":"2024-04-24T18:08:55.894711+0000","last_clean":"2024-04-24T18:08:55.894711+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:55.894711+0000","last_undegraded":"2024-04-24T18:08:55.894711+0000","last_fullsized":"2024-04-24T18:08:55.894711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":288,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191354+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:56.191354+0000","last_peered":"2024-04-24T18:08:56.191354+0000","last_clean":"2024-04-24T18:08:56.191354+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:56.191354+0000","last_undegraded":"2024-04-24T18:08:56.191354+0000","last_fullsized":"2024-04-24T18:08:56.191354+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.894092+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:08:55.894092+0000","last_peered":"2024-04-24T18:08:55.894092+0000","last_clean":"2024-04-24T18:08:55.894092+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:08:55.894092+0000","last_undegraded":"2024-04-24T18:08:55.894092+0000","last_fullsized":"2024-04-24T18:08:55.894092+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"97'764","reported_seq":8211,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.008859+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:08:56.008859+0000","last_peered":"2024-04-24T18:08:56.008859+0000","last_clean":"2024-04-24T18:08:56.008859+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:08:56.008859+0000","last_undegraded":"2024-04-24T18:08:56.008859+0000","last_fullsized":"2024-04-24T18:08:56.008859+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":43062717,"num_objects":98,"num_object_clones":16,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":58,"num_read":11080,"num_read_kb":29375,"num_write":966,"num_write_kb":21026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"97'494","reported_seq":5479,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898225+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:08:55.898225+0000","last_peered":"2024-04-24T18:08:55.898225+0000","last_clean":"2024-04-24T18:08:55.898225+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:08:55.898225+0000","last_undegraded":"2024-04-24T18:08:55.898225+0000","last_fullsized":"2024-04-24T18:08:55.898225+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":48776479,"num_objects":116,"num_object_clones":18,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":83,"num_read":6300,"num_read_kb":29404,"num_write":416,"num_write_kb":23341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.899385+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:55.899385+0000","last_peered":"2024-04-24T18:08:55.899385+0000","last_clean":"2024-04-24T18:08:55.899385+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:55.899385+0000","last_undegraded":"2024-04-24T18:08:55.899385+0000","last_fullsized":"2024-04-24T18:08:55.899385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"99'535","reported_seq":4575,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.958671+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:08:56.958671+0000","last_peered":"2024-04-24T18:08:56.958671+0000","last_clean":"2024-04-24T18:08:56.958671+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:08:56.958671+0000","last_undegraded":"2024-04-24T18:08:56.958671+0000","last_fullsized":"2024-04-24T18:08:56.958671+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":40019230,"num_objects":107,"num_object_clones":19,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":62,"num_read":5850,"num_read_kb":14786,"num_write":559,"num_write_kb":7238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900081+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:55.900081+0000","last_peered":"2024-04-24T18:08:55.900081+0000","last_clean":"2024-04-24T18:08:55.900081+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:55.900081+0000","last_undegraded":"2024-04-24T18:08:55.900081+0000","last_fullsized":"2024-04-24T18:08:55.900081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"97'455","reported_seq":8070,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.917598+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:08:56.917598+0000","last_peered":"2024-04-24T18:08:56.917598+0000","last_clean":"2024-04-24T18:08:56.917598+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:08:56.917598+0000","last_undegraded":"2024-04-24T18:08:56.917598+0000","last_fullsized":"2024-04-24T18:08:56.917598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":44322609,"num_objects":124,"num_object_clones":23,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":77,"num_read":8769,"num_read_kb":33538,"num_write":423,"num_write_kb":15372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"97'496","reported_seq":5791,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898074+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:08:55.898074+0000","last_peered":"2024-04-24T18:08:55.898074+0000","last_clean":"2024-04-24T18:08:55.898074+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:08:55.898074+0000","last_undegraded":"2024-04-24T18:08:55.898074+0000","last_fullsized":"2024-04-24T18:08:55.898074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":46300149,"num_objects":117,"num_object_clones":19,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":72,"num_read":6019,"num_read_kb":17989,"num_write":543,"num_write_kb":12639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"97'554","reported_seq":1979,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.900408+0000","last_change":"2024-04-24T18:08:55.900231+0000","last_active":"2024-04-24T18:08:55.900030+0000","last_peered":"2024-04-24T18:08:55.671491+0000","last_clean":"2024-04-24T18:08:18.932949+0000","last_became_active":"2024-04-24T18:08:54.974371+0000","last_became_peered":"2024-04-24T18:08:54.974371+0000","last_unstale":"2024-04-24T18:08:55.900408+0000","last_undegraded":"2024-04-24T18:08:55.900408+0000","last_fullsized":"2024-04-24T18:08:55.900408+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:44:42.192974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":23653650,"num_objects":84,"num_object_clones":7,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":59,"num_read":2223,"num_read_kb":7448,"num_write":638,"num_write_kb":10347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"97'493","reported_seq":4184,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.000383+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:08:56.000383+0000","last_peered":"2024-04-24T18:08:56.000383+0000","last_clean":"2024-04-24T18:08:56.000383+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:08:56.000383+0000","last_undegraded":"2024-04-24T18:08:56.000383+0000","last_fullsized":"2024-04-24T18:08:56.000383+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":24426121,"num_objects":95,"num_object_clones":15,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":63,"num_read":6412,"num_read_kb":13446,"num_write":515,"num_write_kb":10233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"97'440","reported_seq":5523,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.928037+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:08:56.928037+0000","last_peered":"2024-04-24T18:08:56.928037+0000","last_clean":"2024-04-24T18:08:56.928037+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:08:56.928037+0000","last_undegraded":"2024-04-24T18:08:56.928037+0000","last_fullsized":"2024-04-24T18:08:56.928037+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":39221197,"num_objects":94,"num_object_clones":15,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":61,"num_read":6301,"num_read_kb":26476,"num_write":425,"num_write_kb":27931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"94'468","reported_seq":6852,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.911816+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:08:56.911816+0000","last_peered":"2024-04-24T18:08:56.911816+0000","last_clean":"2024-04-24T18:08:56.911816+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:08:56.911816+0000","last_undegraded":"2024-04-24T18:08:56.911816+0000","last_fullsized":"2024-04-24T18:08:56.911816+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":44049617,"num_objects":132,"num_object_clones":24,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":84,"num_read":6868,"num_read_kb":16837,"num_write":387,"num_write_kb":5434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"97'434","reported_seq":5590,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.938722+0000","last_change":"2024-04-24T18:08:56.920936+0000","last_active":"2024-04-24T18:08:56.938722+0000","last_peered":"2024-04-24T18:08:56.938722+0000","last_clean":"2024-04-24T18:08:56.938722+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:08:56.938722+0000","last_undegraded":"2024-04-24T18:08:56.938722+0000","last_fullsized":"2024-04-24T18:08:56.938722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":42388138,"num_objects":110,"num_object_clones":24,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":63,"num_read":5566,"num_read_kb":14066,"num_write":415,"num_write_kb":12876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"99'643","reported_seq":3498,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894417+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:08:56.894417+0000","last_peered":"2024-04-24T18:08:56.894417+0000","last_clean":"2024-04-24T18:08:56.894417+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:08:56.894417+0000","last_undegraded":"2024-04-24T18:08:56.894417+0000","last_fullsized":"2024-04-24T18:08:56.894417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":32472669,"num_objects":113,"num_object_clones":15,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":70,"num_read":5542,"num_read_kb":17226,"num_write":728,"num_write_kb":7460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"94'462","reported_seq":5843,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.899349+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:08:55.899349+0000","last_peered":"2024-04-24T18:08:55.899349+0000","last_clean":"2024-04-24T18:08:55.899349+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:08:55.899349+0000","last_undegraded":"2024-04-24T18:08:55.899349+0000","last_fullsized":"2024-04-24T18:08:55.899349+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":28645466,"num_objects":125,"num_object_clones":24,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":79,"num_read":6684,"num_read_kb":15620,"num_write":391,"num_write_kb":7897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"97'584","reported_seq":7578,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.014510+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:08:56.014510+0000","last_peered":"2024-04-24T18:08:56.014510+0000","last_clean":"2024-04-24T18:08:56.014510+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:08:56.014510+0000","last_undegraded":"2024-04-24T18:08:56.014510+0000","last_fullsized":"2024-04-24T18:08:56.014510+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":92624525,"num_objects":129,"num_object_clones":26,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":83,"num_read":8296,"num_read_kb":46021,"num_write":663,"num_write_kb":29350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"94'502","reported_seq":5426,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.897040+0000","last_change":"2024-04-24T18:08:56.897040+0000","last_active":"2024-04-24T18:08:56.897040+0000","last_peered":"2024-04-24T18:08:56.897040+0000","last_clean":"2024-04-24T18:08:56.897040+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:08:56.897040+0000","last_undegraded":"2024-04-24T18:08:56.897040+0000","last_fullsized":"2024-04-24T18:08:56.897040+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":25564751,"num_objects":120,"num_object_clones":16,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":82,"num_read":9144,"num_read_kb":29322,"num_write":422,"num_write_kb":8077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"97'840","reported_seq":7979,"reported_epoch":99,"state":"peering","last_fresh":"2024-04-24T18:08:56.898707+0000","last_change":"2024-04-24T18:08:56.899109+0000","last_active":"2024-04-24T18:08:56.898674+0000","last_peered":"2024-04-24T18:08:56.161342+0000","last_clean":"2024-04-24T18:08:20.733644+0000","last_became_active":"2024-04-24T18:08:54.984176+0000","last_became_peered":"2024-04-24T18:08:54.984176+0000","last_unstale":"2024-04-24T18:08:56.898707+0000","last_undegraded":"2024-04-24T18:08:56.898707+0000","last_fullsized":"2024-04-24T18:08:56.898707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:07.099674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":43298546,"num_objects":110,"num_object_clones":26,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":67,"num_read":9613,"num_read_kb":21463,"num_write":1186,"num_write_kb":22201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"97'454","reported_seq":6563,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.003799+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:08:56.003799+0000","last_peered":"2024-04-24T18:08:56.003799+0000","last_clean":"2024-04-24T18:08:56.003799+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:08:56.003799+0000","last_undegraded":"2024-04-24T18:08:56.003799+0000","last_fullsized":"2024-04-24T18:08:56.003799+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":48303822,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":57,"num_read":8379,"num_read_kb":22105,"num_write":399,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"97'425","reported_seq":4884,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.013828+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:08:56.013828+0000","last_peered":"2024-04-24T18:08:56.013828+0000","last_clean":"2024-04-24T18:08:56.013828+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:08:56.013828+0000","last_undegraded":"2024-04-24T18:08:56.013828+0000","last_fullsized":"2024-04-24T18:08:56.013828+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":19979170,"num_objects":107,"num_object_clones":20,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":69,"num_read":6414,"num_read_kb":11795,"num_write":358,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"99'930","reported_seq":3744,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894424+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:08:56.894424+0000","last_peered":"2024-04-24T18:08:56.894424+0000","last_clean":"2024-04-24T18:08:56.894424+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:08:56.894424+0000","last_undegraded":"2024-04-24T18:08:56.894424+0000","last_fullsized":"2024-04-24T18:08:56.894424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":930,"log_dups_size":0,"ondisk_log_size":930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":56197220,"num_objects":128,"num_object_clones":25,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":81,"num_read":5751,"num_read_kb":28663,"num_write":1227,"num_write_kb":15759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1328587432,"num_objects":3702,"num_object_clones":673,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":2328,"num_read":226989,"num_read_kb":733041,"num_write":20848,"num_write_kb":471327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70193152,"data_stored":666267624,"data_compressed":13212412,"data_compressed_allocated":43180032,"data_compressed_original":642892040,"omap_allocated":9738,"internal_metadata":0},"log_size":18537,"ondisk_log_size":18537,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317532,"kb_used_data":7868,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94054308,"statfs":{"total":96636764160,"available":96311611392,"internally_reserved":0,"allocated":8056832,"data_stored":68522011,"data_compressed":1018862,"data_compressed_allocated":4190208,"data_compressed_original":65758884,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.375}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.437},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.30099999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.33300000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.42599999999999999}]}]},{"osd":6,"up_from":95,"seq":408021893122,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310184,"kb_used_data":7452,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94061656,"statfs":{"total":96636764160,"available":96319135744,"internally_reserved":0,"allocated":7630848,"data_stored":62744965,"data_compressed":1452463,"data_compressed_allocated":4415488,"data_compressed_original":60366517,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323224,"kb_used_data":13580,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94048616,"statfs":{"total":96636764160,"available":96305782784,"internally_reserved":0,"allocated":13905920,"data_stored":98128349,"data_compressed":1986573,"data_compressed_allocated":6615040,"data_compressed_original":92426435,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.15,"5min":14.15,"15min":14.15},"last":0.311}]},{"osd":2,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.36199999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":14.089,"5min":14.089,"15min":14.089},"last":0.432},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.45500000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":14.051,"5min":14.051,"15min":14.051},"last":0.501},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.45000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320280,"kb_used_data":6540,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94051560,"statfs":{"total":96636764160,"available":96308797440,"internally_reserved":0,"allocated":6696960,"data_stored":60726874,"data_compressed":1053017,"data_compressed_allocated":3735552,"data_compressed_original":58676108,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.29799999999999999},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":26.936,"5min":26.936,"15min":26.936},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":14.371,"5min":14.371,"15min":14.371},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.973000000000001,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.414,"5min":14.414,"15min":14.414},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":14.452,"5min":14.452,"15min":14.452},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":14.486000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.57499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313664,"kb_used_data":4020,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058176,"statfs":{"total":96636764160,"available":96315572224,"internally_reserved":0,"allocated":4116480,"data_stored":24654978,"data_compressed":524522,"data_compressed_allocated":1728512,"data_compressed_original":23053518,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.23899999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.26900000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.37},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.248}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.499}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320680,"kb_used_data":15136,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94051160,"statfs":{"total":96636764160,"available":96308387840,"internally_reserved":0,"allocated":15499264,"data_stored":142743749,"data_compressed":2373991,"data_compressed_allocated":8704000,"data_compressed_original":137212312,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.371},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.504}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64200000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.62},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.754},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313732,"kb_used_data":12276,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94058108,"statfs":{"total":96636764160,"available":96315502592,"internally_reserved":0,"allocated":12570624,"data_stored":121128547,"data_compressed":2722834,"data_compressed_allocated":7471104,"data_compressed_original":116902284,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.441}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":2.5019999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":1.373},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.629}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323712,"kb_used_data":9968,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94048128,"statfs":{"total":96636764160,"available":96305283072,"internally_reserved":0,"allocated":10207232,"data_stored":84056211,"data_compressed":1838523,"data_compressed_allocated":5640192,"data_compressed_original":80508782,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.39000000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.437}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.33600000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.42599999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.41199999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5607424,"data_stored":61064277,"data_compressed":1086854,"data_compressed_allocated":3821568,"data_compressed_original":59593612,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12914688,"data_stored":99716648,"data_compressed":2031230,"data_compressed_allocated":6791168,"data_compressed_original":94589123,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3043328,"data_stored":24998637,"data_compressed":561354,"data_compressed_allocated":1818624,"data_compressed_original":23971022,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14323712,"data_stored":142163648,"data_compressed":2373991,"data_compressed_allocated":8704000,"data_compressed_original":137212312,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11395072,"data_stored":120548446,"data_compressed":2722834,"data_compressed_allocated":7471104,"data_compressed_original":116902284,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9043968,"data_stored":83482366,"data_compressed":1838523,"data_compressed_allocated":5640192,"data_compressed_original":80508782,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7041024,"data_stored":66942060,"data_compressed":1587428,"data_compressed_allocated":4780032,"data_compressed_original":64929461,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6823936,"data_stored":67351542,"data_compressed":1010198,"data_compressed_allocated":4153344,"data_compressed_original":65185444,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:00.578 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:09:00.579 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:00.579 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:00.579 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:00.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:00.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:00.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:00.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:00.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:00.580 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:00.586 INFO:teuthology.orchestra.run.smithi049.stdout:5395 write 0xc80e3c3 thru 0xc81823d (0x9e7b bytes) 2024-04-24T18:09:00.601 INFO:teuthology.orchestra.run.smithi049.stdout:5398 write 0x1e27f63 thru 0x1e32bc1 (0xac5f bytes) 2024-04-24T18:09:00.613 INFO:teuthology.orchestra.run.smithi049.stdout:5401 writesame 0x7a12edd thru 0x7a14fa1 data_size 0x1(0x20c5 bytes) 2024-04-24T18:09:00.622 INFO:teuthology.orchestra.run.smithi049.stdout:5402 read 0x5134082 thru 0x51416bb (0xd63a bytes) 2024-04-24T18:09:00.623 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:00.623 INFO:teuthology.orchestra.run.smithi049.stdout:5404 write 0x97abbc8 thru 0x97afc91 (0x40ca bytes) 2024-04-24T18:09:00.629 INFO:teuthology.orchestra.run.smithi049.stdout:5405 write 0x5b5c1bf thru 0x5b5f0e7 (0x2f29 bytes) 2024-04-24T18:09:00.635 INFO:teuthology.orchestra.run.smithi049.stdout:5406 write 0x6edd028 thru 0x6eeb062 (0xe03b bytes) 2024-04-24T18:09:00.647 INFO:teuthology.orchestra.run.smithi049.stdout:5408 write 0x33f47de thru 0x33fa0bf (0x58e2 bytes) 2024-04-24T18:09:00.656 INFO:teuthology.orchestra.run.smithi049.stdout:5410 write 0x600ea99 thru 0x601a560 (0xbac8 bytes) 2024-04-24T18:09:00.659 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:00.667+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:00.666 INFO:teuthology.orchestra.run.smithi049.stdout:5411 write 0xb8f35aa thru 0xb900ac0 (0xd517 bytes) 2024-04-24T18:09:00.680 INFO:teuthology.orchestra.run.smithi049.stdout:5412 writesame 0xc649d89 thru 0xc65819b data_size 0x13(0xe413 bytes) 2024-04-24T18:09:00.689 INFO:teuthology.orchestra.run.smithi049.stdout:5413 punch from 0x98aa38f to 0x98b82e2, (0xdf53 bytes) 2024-04-24T18:09:00.689 INFO:teuthology.orchestra.run.smithi049.stdout:5414 write 0xa8bede3 thru 0xa8c4d1a (0x5f38 bytes) 2024-04-24T18:09:00.697 INFO:teuthology.orchestra.run.smithi049.stdout:5415 read 0x625ae23 thru 0x6267ad7 (0xccb5 bytes) 2024-04-24T18:09:00.698 INFO:teuthology.orchestra.run.smithi049.stdout:5417 writesame 0x5a9e780 thru 0x5aa90d2 data_size 0x1(0xa953 bytes) 2024-04-24T18:09:00.707 INFO:teuthology.orchestra.run.smithi049.stdout:5418 write 0x759d354 thru 0x759fe61 (0x2b0e bytes) 2024-04-24T18:09:00.709 INFO:teuthology.orchestra.run.smithi049.stdout:5419 writesame 0xaa7e3ec thru 0xaa8c02f data_size 0x94(0xdc44 bytes) 2024-04-24T18:09:00.714 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:00.714 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:00.715 INFO:teuthology.orchestra.run.smithi049.stdout:5421 read 0xc6017df thru 0xc605c18 (0x443a bytes) 2024-04-24T18:09:00.722 INFO:teuthology.orchestra.run.smithi049.stdout:5423 writesame 0xd3e6f38 thru 0xd3f1041 data_size 0x3(0xa10a bytes) 2024-04-24T18:09:00.725 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:00.733 INFO:teuthology.orchestra.run.smithi049.stdout:5424 trunc from 0xd3f1042 to 0x51e7bce 2024-04-24T18:09:00.766 INFO:teuthology.orchestra.run.smithi049.stdout:5425 read 0x3fa3f64 thru 0x3fa7053 (0x30f0 bytes) 2024-04-24T18:09:00.766 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:00.766 INFO:teuthology.orchestra.run.smithi049.stdout:5429 writesame 0x2c1a8f1 thru 0x2c289cb data_size 0x1(0xe0db bytes) 2024-04-24T18:09:00.768 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:00.777+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:00.786 INFO:teuthology.orchestra.run.smithi049.stdout:5430 write 0x6af5e77 thru 0x6b04540 (0xe6ca bytes) 2024-04-24T18:09:00.799 INFO:teuthology.orchestra.run.smithi049.stdout:5431 read 0x34c589 thru 0x35448c (0x7f04 bytes) 2024-04-24T18:09:00.806 INFO:teuthology.orchestra.run.smithi049.stdout:5432 trunc from 0x6b04541 to 0x11716ff 2024-04-24T18:09:00.824 INFO:teuthology.orchestra.run.smithi049.stdout:5433 read 0x4b646c thru 0x4b96ea (0x327f bytes) 2024-04-24T18:09:00.825 INFO:teuthology.orchestra.run.smithi049.stdout:5434 trunc from 0x11716ff to 0x733dc1 2024-04-24T18:09:00.851 INFO:teuthology.orchestra.run.smithi049.stdout:5437 write 0x7d5bcba thru 0x7d62550 (0x6897 bytes) 2024-04-24T18:09:00.861 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:00.869+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:00.863 INFO:teuthology.orchestra.run.smithi049.stdout:5438 punch from 0x6a5ca0c to 0x6a6a8c4, (0xdeb8 bytes) 2024-04-24T18:09:00.863 INFO:teuthology.orchestra.run.smithi049.stdout:5441 read 0x13775d3 thru 0x1382fad (0xb9db bytes) 2024-04-24T18:09:00.863 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:00.863 INFO:teuthology.orchestra.run.smithi049.stdout:5444 writesame 0x377a541 thru 0x3784f18 data_size 0x5(0xa9d8 bytes) 2024-04-24T18:09:00.869 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:00.869 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:00.876 INFO:teuthology.orchestra.run.smithi049.stdout:5445 read 0x74e3801 thru 0x74f10dc (0xd8dc bytes) 2024-04-24T18:09:00.876 INFO:teuthology.orchestra.run.smithi049.stdout:5446 write 0x56fd033 thru 0x5709325 (0xc2f3 bytes) 2024-04-24T18:09:00.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:00.892 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:00.892 INFO:teuthology.orchestra.run.smithi049.stdout:5448 writesame 0x57a908b thru 0x57acc5d data_size 0x5(0x3bd3 bytes) 2024-04-24T18:09:00.894 INFO:teuthology.orchestra.run.smithi049.stdout:5449 writesame 0xa3d760 thru 0xa42044 data_size 0x1(0x48e5 bytes) 2024-04-24T18:09:00.900 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:00.900 INFO:teuthology.orchestra.run.smithi049.stdout:5452 read 0x241d179 thru 0x2427e6d (0xacf5 bytes) 2024-04-24T18:09:00.900 INFO:teuthology.orchestra.run.smithi049.stdout:5453 punch from 0x5a5e6a4 to 0x5a66c29, (0x8585 bytes) 2024-04-24T18:09:00.900 INFO:teuthology.orchestra.run.smithi049.stdout:5454 write 0x7d31a69 thru 0x7d33673 (0x1c0b bytes) 2024-04-24T18:09:00.904 INFO:teuthology.orchestra.run.smithi049.stdout:5455 trunc from 0x7d62551 to 0x4ed7ea2 2024-04-24T18:09:00.916 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:09:00.925 INFO:teuthology.orchestra.run.smithi049.stdout:5456 punch from 0x46e5fd2 to 0x46f1ad0, (0xbafe bytes) 2024-04-24T18:09:00.940 INFO:teuthology.orchestra.run.smithi049.stdout:5457 writesame 0xd024700 thru 0xd02e8ff data_size 0xa2(0xa200 bytes) 2024-04-24T18:09:00.940 INFO:teuthology.orchestra.run.smithi049.stdout:5458 read 0x2cc4be6 thru 0x2cd08b1 (0xbccc bytes) 2024-04-24T18:09:00.940 INFO:teuthology.orchestra.run.smithi049.stdout:5459 punch from 0x2cb073b to 0x2cb1dfd, (0x16c2 bytes) 2024-04-24T18:09:00.940 INFO:teuthology.orchestra.run.smithi049.stdout:5460 read 0xa6fb4bc thru 0xa707780 (0xc2c5 bytes) 2024-04-24T18:09:00.940 INFO:teuthology.orchestra.run.smithi049.stdout:5462 read 0x2de65c4 thru 0x2df395f (0xd39c bytes) 2024-04-24T18:09:00.949 INFO:teuthology.orchestra.run.smithi049.stdout:5464 trunc from 0xd02e900 to 0x4430af9 2024-04-24T18:09:00.966 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:00.974+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:00.972 INFO:teuthology.orchestra.run.smithi049.stdout:5465 writesame 0x9dd98a7 thru 0x9de8c3f data_size 0x7b(0xf399 bytes) 2024-04-24T18:09:00.973 INFO:teuthology.orchestra.run.smithi049.stdout:5466 writesame 0x71beb3b thru 0x71cc3f2 data_size 0x14(0xd8b8 bytes) 2024-04-24T18:09:00.993 INFO:teuthology.orchestra.run.smithi049.stdout:5467 write 0xbd83c2b thru 0xbd874d1 (0x38a7 bytes) 2024-04-24T18:09:01.000 INFO:teuthology.orchestra.run.smithi049.stdout:5468 write 0x307300e thru 0x307425d (0x1250 bytes) 2024-04-24T18:09:01.003 INFO:teuthology.orchestra.run.smithi049.stdout:5469 read 0x64d12de thru 0x64da24f (0x8f72 bytes) 2024-04-24T18:09:01.003 INFO:teuthology.orchestra.run.smithi049.stdout:5470 writesame 0x103a6b7 thru 0x103e2d5 data_size 0x1(0x3c1f bytes) 2024-04-24T18:09:01.015 INFO:teuthology.orchestra.run.smithi049.stdout:5471 read 0xad6bc77 thru 0xad6e9d4 (0x2d5e bytes) 2024-04-24T18:09:01.015 INFO:teuthology.orchestra.run.smithi049.stdout:5473 read 0x892fed4 thru 0x893c3a9 (0xc4d6 bytes) 2024-04-24T18:09:01.015 INFO:teuthology.orchestra.run.smithi049.stdout:5474 read 0x1a693f7 thru 0x1a6b75e (0x2368 bytes) 2024-04-24T18:09:01.016 INFO:teuthology.orchestra.run.smithi049.stdout:5475 write 0x501db1a thru 0x501f9cd (0x1eb4 bytes) 2024-04-24T18:09:01.021 INFO:teuthology.orchestra.run.smithi049.stdout:5476 write 0x9c09e70 thru 0x9c13e43 (0x9fd4 bytes) 2024-04-24T18:09:01.034 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:01.035 INFO:teuthology.orchestra.run.smithi049.stdout:5477 writesame 0x4ce9f8a thru 0x4cf5651 data_size 0x4(0xb6c8 bytes) 2024-04-24T18:09:01.038 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:01.048 INFO:teuthology.orchestra.run.smithi049.stdout:5478 writesame 0x2faec7c thru 0x2fbdb71 data_size 0x49(0xeef6 bytes) 2024-04-24T18:09:01.054 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":147,"stamp":"2024-04-24T18:09:00.917855+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1329177800,"num_objects":3706,"num_object_clones":673,"num_object_copies":7412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3704,"num_whiteouts":2330,"num_read":227105,"num_read_kb":733141,"num_write":21007,"num_write_kb":473743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18656,"ondisk_log_size":18656,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2543008,"kb_used_data":76840,"kb_used_omap":43,"kb_used_meta":2466004,"kb_avail":752431712,"statfs":{"total":773094113280,"available":770490073088,"internally_reserved":0,"allocated":78684160,"data_stored":662705684,"data_compressed":12970785,"data_compressed_allocated":42500096,"data_compressed_original":634904840,"omap_allocated":44665,"internal_metadata":2525188487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5582795,"num_objects":108,"num_object_clones":4,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":101,"num_read":10428,"num_read_kb":26078,"num_write":352,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":11389499,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.024774"},"pg_stats":[{"pgid":"4.1f","version":"97'457","reported_seq":5382,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.901318+0000","last_change":"2024-04-24T18:08:55.899667+0000","last_active":"2024-04-24T18:08:55.899445+0000","last_peered":"2024-04-24T18:08:55.671181+0000","last_clean":"2024-04-24T18:08:18.970509+0000","last_became_active":"2024-04-24T18:08:54.974383+0000","last_became_peered":"2024-04-24T18:08:54.974383+0000","last_unstale":"2024-04-24T18:08:55.901318+0000","last_undegraded":"2024-04-24T18:08:55.901318+0000","last_fullsized":"2024-04-24T18:08:55.901318+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:39:05.236590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":48185911,"num_objects":139,"num_object_clones":25,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":94,"num_read":5428,"num_read_kb":27520,"num_write":459,"num_write_kb":11775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"94'1135","reported_seq":5715,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.895044+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:08:55.895044+0000","last_peered":"2024-04-24T18:08:55.895044+0000","last_clean":"2024-04-24T18:08:55.895044+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:08:55.895044+0000","last_undegraded":"2024-04-24T18:08:55.895044+0000","last_fullsized":"2024-04-24T18:08:55.895044+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1135,"log_dups_size":0,"ondisk_log_size":1135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":38392536,"num_objects":139,"num_object_clones":24,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":89,"num_read":9052,"num_read_kb":19486,"num_write":1680,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"99'512","reported_seq":3960,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.935539+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:08:56.935539+0000","last_peered":"2024-04-24T18:08:56.935539+0000","last_clean":"2024-04-24T18:08:56.935539+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:08:56.935539+0000","last_undegraded":"2024-04-24T18:08:56.935539+0000","last_fullsized":"2024-04-24T18:08:56.935539+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":32404686,"num_objects":138,"num_object_clones":25,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":89,"num_read":4444,"num_read_kb":14922,"num_write":490,"num_write_kb":11863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"97'791","reported_seq":7936,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.895918+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:08:55.895918+0000","last_peered":"2024-04-24T18:08:55.895918+0000","last_clean":"2024-04-24T18:08:55.895918+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:08:55.895918+0000","last_undegraded":"2024-04-24T18:08:55.895918+0000","last_fullsized":"2024-04-24T18:08:55.895918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":39279832,"num_objects":108,"num_object_clones":23,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":63,"num_read":9978,"num_read_kb":21879,"num_write":1068,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"97'384","reported_seq":7066,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.927203+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:08:56.927203+0000","last_peered":"2024-04-24T18:08:56.927203+0000","last_clean":"2024-04-24T18:08:56.927203+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:08:56.927203+0000","last_undegraded":"2024-04-24T18:08:56.927203+0000","last_fullsized":"2024-04-24T18:08:56.927203+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":45749737,"num_objects":112,"num_object_clones":24,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":61,"num_read":6776,"num_read_kb":30196,"num_write":366,"num_write_kb":18770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"94'450","reported_seq":10342,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.919230+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:08:56.919230+0000","last_peered":"2024-04-24T18:08:56.919230+0000","last_clean":"2024-04-24T18:08:56.919230+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:08:56.919230+0000","last_undegraded":"2024-04-24T18:08:56.919230+0000","last_fullsized":"2024-04-24T18:08:56.919230+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":41098372,"num_objects":120,"num_object_clones":27,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":68,"num_read":11189,"num_read_kb":27289,"num_write":394,"num_write_kb":11539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"97'545","reported_seq":8037,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900063+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:08:55.900063+0000","last_peered":"2024-04-24T18:08:55.900063+0000","last_clean":"2024-04-24T18:08:55.900063+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:08:55.900063+0000","last_undegraded":"2024-04-24T18:08:55.900063+0000","last_fullsized":"2024-04-24T18:08:55.900063+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":39665781,"num_objects":116,"num_object_clones":19,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":77,"num_read":8848,"num_read_kb":27076,"num_write":565,"num_write_kb":13542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"98'415","reported_seq":8350,"reported_epoch":99,"state":"peering","last_fresh":"2024-04-24T18:08:56.896474+0000","last_change":"2024-04-24T18:08:56.896899+0000","last_active":"2024-04-24T18:08:56.896439+0000","last_peered":"2024-04-24T18:08:56.041651+0000","last_clean":"2024-04-24T18:08:20.733439+0000","last_became_active":"2024-04-24T18:08:54.975461+0000","last_became_peered":"2024-04-24T18:08:54.975461+0000","last_unstale":"2024-04-24T18:08:56.896474+0000","last_undegraded":"2024-04-24T18:08:56.896474+0000","last_fullsized":"2024-04-24T18:08:56.896474+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:02:54.955318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":44496065,"num_objects":103,"num_object_clones":18,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":60,"num_read":8906,"num_read_kb":26427,"num_write":406,"num_write_kb":26815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"97'414","reported_seq":4852,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.002567+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:08:56.002567+0000","last_peered":"2024-04-24T18:08:56.002567+0000","last_clean":"2024-04-24T18:08:56.002567+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:08:56.002567+0000","last_undegraded":"2024-04-24T18:08:56.002567+0000","last_fullsized":"2024-04-24T18:08:56.002567+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":41936818,"num_objects":114,"num_object_clones":26,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":72,"num_read":5026,"num_read_kb":19857,"num_write":381,"num_write_kb":15303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"97'1035","reported_seq":3056,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.008253+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:08:56.008253+0000","last_peered":"2024-04-24T18:08:56.008253+0000","last_clean":"2024-04-24T18:08:56.008253+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:08:56.008253+0000","last_undegraded":"2024-04-24T18:08:56.008253+0000","last_fullsized":"2024-04-24T18:08:56.008253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1035,"log_dups_size":0,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":41063598,"num_objects":122,"num_object_clones":19,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":75,"num_read":6837,"num_read_kb":20698,"num_write":1522,"num_write_kb":17149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-24T18:08:54.963507+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:08:54.963507+0000","last_peered":"2024-04-24T18:08:54.963507+0000","last_clean":"2024-04-24T18:08:54.963507+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:08:54.963507+0000","last_undegraded":"2024-04-24T18:08:54.963507+0000","last_fullsized":"2024-04-24T18:08:54.963507+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-24T18:08:54.963544+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:08:54.963544+0000","last_peered":"2024-04-24T18:08:54.963544+0000","last_clean":"2024-04-24T18:08:54.963544+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:08:54.963544+0000","last_undegraded":"2024-04-24T18:08:54.963544+0000","last_fullsized":"2024-04-24T18:08:54.963544+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"94'499","reported_seq":4408,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.012311+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:08:56.012311+0000","last_peered":"2024-04-24T18:08:56.012311+0000","last_clean":"2024-04-24T18:08:56.012311+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:08:56.012311+0000","last_undegraded":"2024-04-24T18:08:56.012311+0000","last_fullsized":"2024-04-24T18:08:56.012311+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":59713549,"num_objects":122,"num_object_clones":24,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":74,"num_read":4646,"num_read_kb":29075,"num_write":538,"num_write_kb":18441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":88,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898146+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:55.898146+0000","last_peered":"2024-04-24T18:08:55.898146+0000","last_clean":"2024-04-24T18:08:55.898146+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:55.898146+0000","last_undegraded":"2024-04-24T18:08:55.898146+0000","last_fullsized":"2024-04-24T18:08:55.898146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898226+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:55.898226+0000","last_peered":"2024-04-24T18:08:55.898226+0000","last_clean":"2024-04-24T18:08:55.898226+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:55.898226+0000","last_undegraded":"2024-04-24T18:08:55.898226+0000","last_fullsized":"2024-04-24T18:08:55.898226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"99'822","reported_seq":4432,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894848+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:08:56.894848+0000","last_peered":"2024-04-24T18:08:56.894848+0000","last_clean":"2024-04-24T18:08:56.894848+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:08:56.894848+0000","last_undegraded":"2024-04-24T18:08:56.894848+0000","last_fullsized":"2024-04-24T18:08:56.894848+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":29722279,"num_objects":114,"num_object_clones":18,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":82,"num_read":6845,"num_read_kb":12351,"num_write":1025,"num_write_kb":15481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"61'1","reported_seq":89,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900310+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:55.900310+0000","last_peered":"2024-04-24T18:08:55.900310+0000","last_clean":"2024-04-24T18:08:55.900310+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:55.900310+0000","last_undegraded":"2024-04-24T18:08:55.900310+0000","last_fullsized":"2024-04-24T18:08:55.900310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:06.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191347+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:56.191347+0000","last_peered":"2024-04-24T18:08:56.191347+0000","last_clean":"2024-04-24T18:08:56.191347+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:56.191347+0000","last_undegraded":"2024-04-24T18:08:56.191347+0000","last_fullsized":"2024-04-24T18:08:56.191347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"97'733","reported_seq":6276,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.904960+0000","last_change":"2024-04-24T18:08:55.901049+0000","last_active":"2024-04-24T18:08:55.900833+0000","last_peered":"2024-04-24T18:08:55.613013+0000","last_clean":"2024-04-24T18:08:19.722254+0000","last_became_active":"2024-04-24T18:08:54.974371+0000","last_became_peered":"2024-04-24T18:08:54.974371+0000","last_unstale":"2024-04-24T18:08:55.904960+0000","last_undegraded":"2024-04-24T18:08:55.904960+0000","last_fullsized":"2024-04-24T18:08:55.904960+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:09.259454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":53469738,"num_objects":138,"num_object_clones":21,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":88,"num_read":8268,"num_read_kb":36769,"num_write":822,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.893935+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:55.893935+0000","last_peered":"2024-04-24T18:08:55.893935+0000","last_clean":"2024-04-24T18:08:55.893935+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:55.893935+0000","last_undegraded":"2024-04-24T18:08:55.893935+0000","last_fullsized":"2024-04-24T18:08:55.893935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191290+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:56.191290+0000","last_peered":"2024-04-24T18:08:56.191290+0000","last_clean":"2024-04-24T18:08:56.191290+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:56.191290+0000","last_undegraded":"2024-04-24T18:08:56.191290+0000","last_fullsized":"2024-04-24T18:08:56.191290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"94'372","reported_seq":4298,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.992256+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:08:55.992256+0000","last_peered":"2024-04-24T18:08:55.992256+0000","last_clean":"2024-04-24T18:08:55.992256+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:08:55.992256+0000","last_undegraded":"2024-04-24T18:08:55.992256+0000","last_fullsized":"2024-04-24T18:08:55.992256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":30102454,"num_objects":113,"num_object_clones":28,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":66,"num_read":5535,"num_read_kb":23916,"num_write":471,"num_write_kb":12527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'1","reported_seq":89,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.894711+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:55.894711+0000","last_peered":"2024-04-24T18:08:55.894711+0000","last_clean":"2024-04-24T18:08:55.894711+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:55.894711+0000","last_undegraded":"2024-04-24T18:08:55.894711+0000","last_fullsized":"2024-04-24T18:08:55.894711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":288,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191354+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:56.191354+0000","last_peered":"2024-04-24T18:08:56.191354+0000","last_clean":"2024-04-24T18:08:56.191354+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:56.191354+0000","last_undegraded":"2024-04-24T18:08:56.191354+0000","last_fullsized":"2024-04-24T18:08:56.191354+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.894092+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:08:55.894092+0000","last_peered":"2024-04-24T18:08:55.894092+0000","last_clean":"2024-04-24T18:08:55.894092+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:08:55.894092+0000","last_undegraded":"2024-04-24T18:08:55.894092+0000","last_fullsized":"2024-04-24T18:08:55.894092+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"97'764","reported_seq":8211,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.008859+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:08:56.008859+0000","last_peered":"2024-04-24T18:08:56.008859+0000","last_clean":"2024-04-24T18:08:56.008859+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:08:56.008859+0000","last_undegraded":"2024-04-24T18:08:56.008859+0000","last_fullsized":"2024-04-24T18:08:56.008859+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":43062717,"num_objects":98,"num_object_clones":16,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":58,"num_read":11080,"num_read_kb":29375,"num_write":966,"num_write_kb":21026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"97'494","reported_seq":5479,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898225+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:08:55.898225+0000","last_peered":"2024-04-24T18:08:55.898225+0000","last_clean":"2024-04-24T18:08:55.898225+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:08:55.898225+0000","last_undegraded":"2024-04-24T18:08:55.898225+0000","last_fullsized":"2024-04-24T18:08:55.898225+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":48776479,"num_objects":116,"num_object_clones":18,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":83,"num_read":6300,"num_read_kb":29404,"num_write":416,"num_write_kb":23341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.899385+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:55.899385+0000","last_peered":"2024-04-24T18:08:55.899385+0000","last_clean":"2024-04-24T18:08:55.899385+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:55.899385+0000","last_undegraded":"2024-04-24T18:08:55.899385+0000","last_fullsized":"2024-04-24T18:08:55.899385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"99'535","reported_seq":4575,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.958671+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:08:56.958671+0000","last_peered":"2024-04-24T18:08:56.958671+0000","last_clean":"2024-04-24T18:08:56.958671+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:08:56.958671+0000","last_undegraded":"2024-04-24T18:08:56.958671+0000","last_fullsized":"2024-04-24T18:08:56.958671+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":40019230,"num_objects":107,"num_object_clones":19,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":62,"num_read":5850,"num_read_kb":14786,"num_write":559,"num_write_kb":7238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900081+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:55.900081+0000","last_peered":"2024-04-24T18:08:55.900081+0000","last_clean":"2024-04-24T18:08:55.900081+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:55.900081+0000","last_undegraded":"2024-04-24T18:08:55.900081+0000","last_fullsized":"2024-04-24T18:08:55.900081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"97'455","reported_seq":8070,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.917598+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:08:56.917598+0000","last_peered":"2024-04-24T18:08:56.917598+0000","last_clean":"2024-04-24T18:08:56.917598+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:08:56.917598+0000","last_undegraded":"2024-04-24T18:08:56.917598+0000","last_fullsized":"2024-04-24T18:08:56.917598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":44322609,"num_objects":124,"num_object_clones":23,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":77,"num_read":8769,"num_read_kb":33538,"num_write":423,"num_write_kb":15372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"97'496","reported_seq":5791,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898074+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:08:55.898074+0000","last_peered":"2024-04-24T18:08:55.898074+0000","last_clean":"2024-04-24T18:08:55.898074+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:08:55.898074+0000","last_undegraded":"2024-04-24T18:08:55.898074+0000","last_fullsized":"2024-04-24T18:08:55.898074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":46300149,"num_objects":117,"num_object_clones":19,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":72,"num_read":6019,"num_read_kb":17989,"num_write":543,"num_write_kb":12639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"97'554","reported_seq":1979,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.900408+0000","last_change":"2024-04-24T18:08:55.900231+0000","last_active":"2024-04-24T18:08:55.900030+0000","last_peered":"2024-04-24T18:08:55.671491+0000","last_clean":"2024-04-24T18:08:18.932949+0000","last_became_active":"2024-04-24T18:08:54.974371+0000","last_became_peered":"2024-04-24T18:08:54.974371+0000","last_unstale":"2024-04-24T18:08:55.900408+0000","last_undegraded":"2024-04-24T18:08:55.900408+0000","last_fullsized":"2024-04-24T18:08:55.900408+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:44:42.192974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":23653650,"num_objects":84,"num_object_clones":7,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":59,"num_read":2223,"num_read_kb":7448,"num_write":638,"num_write_kb":10347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"97'493","reported_seq":4184,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.000383+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:08:56.000383+0000","last_peered":"2024-04-24T18:08:56.000383+0000","last_clean":"2024-04-24T18:08:56.000383+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:08:56.000383+0000","last_undegraded":"2024-04-24T18:08:56.000383+0000","last_fullsized":"2024-04-24T18:08:56.000383+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":24426121,"num_objects":95,"num_object_clones":15,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":63,"num_read":6412,"num_read_kb":13446,"num_write":515,"num_write_kb":10233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"97'440","reported_seq":5523,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.928037+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:08:56.928037+0000","last_peered":"2024-04-24T18:08:56.928037+0000","last_clean":"2024-04-24T18:08:56.928037+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:08:56.928037+0000","last_undegraded":"2024-04-24T18:08:56.928037+0000","last_fullsized":"2024-04-24T18:08:56.928037+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":39221197,"num_objects":94,"num_object_clones":15,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":61,"num_read":6301,"num_read_kb":26476,"num_write":425,"num_write_kb":27931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"94'468","reported_seq":6852,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.911816+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:08:56.911816+0000","last_peered":"2024-04-24T18:08:56.911816+0000","last_clean":"2024-04-24T18:08:56.911816+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:08:56.911816+0000","last_undegraded":"2024-04-24T18:08:56.911816+0000","last_fullsized":"2024-04-24T18:08:56.911816+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":44049617,"num_objects":132,"num_object_clones":24,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":84,"num_read":6868,"num_read_kb":16837,"num_write":387,"num_write_kb":5434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"97'434","reported_seq":5590,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.938722+0000","last_change":"2024-04-24T18:08:56.920936+0000","last_active":"2024-04-24T18:08:56.938722+0000","last_peered":"2024-04-24T18:08:56.938722+0000","last_clean":"2024-04-24T18:08:56.938722+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:08:56.938722+0000","last_undegraded":"2024-04-24T18:08:56.938722+0000","last_fullsized":"2024-04-24T18:08:56.938722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":42388138,"num_objects":110,"num_object_clones":24,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":63,"num_read":5566,"num_read_kb":14066,"num_write":415,"num_write_kb":12876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"99'643","reported_seq":3498,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894417+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:08:56.894417+0000","last_peered":"2024-04-24T18:08:56.894417+0000","last_clean":"2024-04-24T18:08:56.894417+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:08:56.894417+0000","last_undegraded":"2024-04-24T18:08:56.894417+0000","last_fullsized":"2024-04-24T18:08:56.894417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":32472669,"num_objects":113,"num_object_clones":15,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":70,"num_read":5542,"num_read_kb":17226,"num_write":728,"num_write_kb":7460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"94'462","reported_seq":5843,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.899349+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:08:55.899349+0000","last_peered":"2024-04-24T18:08:55.899349+0000","last_clean":"2024-04-24T18:08:55.899349+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:08:55.899349+0000","last_undegraded":"2024-04-24T18:08:55.899349+0000","last_fullsized":"2024-04-24T18:08:55.899349+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":28645466,"num_objects":125,"num_object_clones":24,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":79,"num_read":6684,"num_read_kb":15620,"num_write":391,"num_write_kb":7897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"97'584","reported_seq":7578,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.014510+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:08:56.014510+0000","last_peered":"2024-04-24T18:08:56.014510+0000","last_clean":"2024-04-24T18:08:56.014510+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:08:56.014510+0000","last_undegraded":"2024-04-24T18:08:56.014510+0000","last_fullsized":"2024-04-24T18:08:56.014510+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":92624525,"num_objects":129,"num_object_clones":26,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":83,"num_read":8296,"num_read_kb":46021,"num_write":663,"num_write_kb":29350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"94'502","reported_seq":5426,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.897040+0000","last_change":"2024-04-24T18:08:56.897040+0000","last_active":"2024-04-24T18:08:56.897040+0000","last_peered":"2024-04-24T18:08:56.897040+0000","last_clean":"2024-04-24T18:08:56.897040+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:08:56.897040+0000","last_undegraded":"2024-04-24T18:08:56.897040+0000","last_fullsized":"2024-04-24T18:08:56.897040+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":25564751,"num_objects":120,"num_object_clones":16,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":82,"num_read":9144,"num_read_kb":29322,"num_write":422,"num_write_kb":8077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"97'840","reported_seq":7979,"reported_epoch":99,"state":"peering","last_fresh":"2024-04-24T18:08:56.898707+0000","last_change":"2024-04-24T18:08:56.899109+0000","last_active":"2024-04-24T18:08:56.898674+0000","last_peered":"2024-04-24T18:08:56.161342+0000","last_clean":"2024-04-24T18:08:20.733644+0000","last_became_active":"2024-04-24T18:08:54.984176+0000","last_became_peered":"2024-04-24T18:08:54.984176+0000","last_unstale":"2024-04-24T18:08:56.898707+0000","last_undegraded":"2024-04-24T18:08:56.898707+0000","last_fullsized":"2024-04-24T18:08:56.898707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:07.099674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":43298546,"num_objects":110,"num_object_clones":26,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":67,"num_read":9613,"num_read_kb":21463,"num_write":1186,"num_write_kb":22201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"97'454","reported_seq":6563,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.003799+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:08:56.003799+0000","last_peered":"2024-04-24T18:08:56.003799+0000","last_clean":"2024-04-24T18:08:56.003799+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:08:56.003799+0000","last_undegraded":"2024-04-24T18:08:56.003799+0000","last_fullsized":"2024-04-24T18:08:56.003799+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":48303822,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":57,"num_read":8379,"num_read_kb":22105,"num_write":399,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"97'425","reported_seq":4884,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.013828+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:08:56.013828+0000","last_peered":"2024-04-24T18:08:56.013828+0000","last_clean":"2024-04-24T18:08:56.013828+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:08:56.013828+0000","last_undegraded":"2024-04-24T18:08:56.013828+0000","last_fullsized":"2024-04-24T18:08:56.013828+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":19979170,"num_objects":107,"num_object_clones":20,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":69,"num_read":6414,"num_read_kb":11795,"num_write":358,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"99'930","reported_seq":3744,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894424+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:08:56.894424+0000","last_peered":"2024-04-24T18:08:56.894424+0000","last_clean":"2024-04-24T18:08:56.894424+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:08:56.894424+0000","last_undegraded":"2024-04-24T18:08:56.894424+0000","last_fullsized":"2024-04-24T18:08:56.894424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":930,"log_dups_size":0,"ondisk_log_size":930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":56197220,"num_objects":128,"num_object_clones":25,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":81,"num_read":5751,"num_read_kb":28663,"num_write":1227,"num_write_kb":15759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1328587432,"num_objects":3702,"num_object_clones":673,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":2328,"num_read":226989,"num_read_kb":733041,"num_write":20848,"num_write_kb":471327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70193152,"data_stored":666267624,"data_compressed":13212412,"data_compressed_allocated":43180032,"data_compressed_original":642892040,"omap_allocated":9738,"internal_metadata":0},"log_size":18537,"ondisk_log_size":18537,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317532,"kb_used_data":7868,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94054308,"statfs":{"total":96636764160,"available":96311611392,"internally_reserved":0,"allocated":8056832,"data_stored":68522011,"data_compressed":1018862,"data_compressed_allocated":4190208,"data_compressed_original":65758884,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.375}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.437},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.30099999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.33300000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.42599999999999999}]}]},{"osd":6,"up_from":95,"seq":408021893122,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310184,"kb_used_data":7452,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94061656,"statfs":{"total":96636764160,"available":96319135744,"internally_reserved":0,"allocated":7630848,"data_stored":62744965,"data_compressed":1452463,"data_compressed_allocated":4415488,"data_compressed_original":60366517,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323224,"kb_used_data":13580,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94048616,"statfs":{"total":96636764160,"available":96305782784,"internally_reserved":0,"allocated":13905920,"data_stored":98128349,"data_compressed":1986573,"data_compressed_allocated":6615040,"data_compressed_original":92426435,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.15,"5min":14.15,"15min":14.15},"last":0.311}]},{"osd":2,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.36199999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":14.089,"5min":14.089,"15min":14.089},"last":0.432},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.45500000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":14.051,"5min":14.051,"15min":14.051},"last":0.501},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.45000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320280,"kb_used_data":6540,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94051560,"statfs":{"total":96636764160,"available":96308797440,"internally_reserved":0,"allocated":6696960,"data_stored":60726874,"data_compressed":1053017,"data_compressed_allocated":3735552,"data_compressed_original":58676108,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.29799999999999999},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":26.936,"5min":26.936,"15min":26.936},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":14.371,"5min":14.371,"15min":14.371},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.973000000000001,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.414,"5min":14.414,"15min":14.414},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":14.452,"5min":14.452,"15min":14.452},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":14.486000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.57499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313664,"kb_used_data":4020,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058176,"statfs":{"total":96636764160,"available":96315572224,"internally_reserved":0,"allocated":4116480,"data_stored":24654978,"data_compressed":524522,"data_compressed_allocated":1728512,"data_compressed_original":23053518,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.23899999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.26900000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.37},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.248}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.499}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320680,"kb_used_data":15136,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94051160,"statfs":{"total":96636764160,"available":96308387840,"internally_reserved":0,"allocated":15499264,"data_stored":142743749,"data_compressed":2373991,"data_compressed_allocated":8704000,"data_compressed_original":137212312,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.371},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.504}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64200000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.62},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.754},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313732,"kb_used_data":12276,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94058108,"statfs":{"total":96636764160,"available":96315502592,"internally_reserved":0,"allocated":12570624,"data_stored":121128547,"data_compressed":2722834,"data_compressed_allocated":7471104,"data_compressed_original":116902284,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.441}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":2.5019999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":1.373},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.629}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323712,"kb_used_data":9968,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94048128,"statfs":{"total":96636764160,"available":96305283072,"internally_reserved":0,"allocated":10207232,"data_stored":84056211,"data_compressed":1838523,"data_compressed_allocated":5640192,"data_compressed_original":80508782,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.39000000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.437}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.33600000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.42599999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.41199999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5607424,"data_stored":61064277,"data_compressed":1086854,"data_compressed_allocated":3821568,"data_compressed_original":59593612,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12914688,"data_stored":99716648,"data_compressed":2031230,"data_compressed_allocated":6791168,"data_compressed_original":94589123,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3043328,"data_stored":24998637,"data_compressed":561354,"data_compressed_allocated":1818624,"data_compressed_original":23971022,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14323712,"data_stored":142163648,"data_compressed":2373991,"data_compressed_allocated":8704000,"data_compressed_original":137212312,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11395072,"data_stored":120548446,"data_compressed":2722834,"data_compressed_allocated":7471104,"data_compressed_original":116902284,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9043968,"data_stored":83482366,"data_compressed":1838523,"data_compressed_allocated":5640192,"data_compressed_original":80508782,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7041024,"data_stored":66942060,"data_compressed":1587428,"data_compressed_allocated":4780032,"data_compressed_original":64929461,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6823936,"data_stored":67351542,"data_compressed":1010198,"data_compressed_allocated":4153344,"data_compressed_original":65185444,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:01.057 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2024-04-24T18:09:01.061 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:01.061 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:01.061 INFO:teuthology.orchestra.run.smithi049.stdout:5479 trunc from 0xbd874d2 to 0xd1e2e2f 2024-04-24T18:09:01.063 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:01.071+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:01.067 INFO:teuthology.orchestra.run.smithi049.stdout:5481 read 0x39e22e6 thru 0x39e37c2 (0x14dd bytes) 2024-04-24T18:09:01.067 INFO:teuthology.orchestra.run.smithi049.stdout:5483 read 0x4c500bd thru 0x4c5763a (0x757e bytes) 2024-04-24T18:09:01.072 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:01.081 INFO:teuthology.orchestra.run.smithi049.stdout:5484 trunc from 0xd1e2e2f to 0x18540e6 2024-04-24T18:09:01.132 INFO:teuthology.orchestra.run.smithi049.stdout:5487 writesame 0xbed8700 thru 0xbed8e32 data_size 0x1(0x733 bytes) 2024-04-24T18:09:01.132 INFO:teuthology.orchestra.run.smithi049.stdout:5488 write 0xbb28acc thru 0xbb2d2ad (0x47e2 bytes) 2024-04-24T18:09:01.142 INFO:teuthology.orchestra.run.smithi049.stdout:5489 writesame 0xab7d7fb thru 0xab7f981 data_size 0x1(0x2187 bytes) 2024-04-24T18:09:01.150 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.150 INFO:teuthology.orchestra.run.smithi049.stdout:5491 write 0x438f9b5 thru 0x438fffc (0x648 bytes) 2024-04-24T18:09:01.153 INFO:teuthology.orchestra.run.smithi049.stdout:5492 read 0x75e5c65 thru 0x75f3f8f (0xe32b bytes) 2024-04-24T18:09:01.153 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.153 INFO:teuthology.orchestra.run.smithi049.stdout:5494 punch from 0x294d14a to 0x295498d, (0x7843 bytes) 2024-04-24T18:09:01.153 INFO:teuthology.orchestra.run.smithi049.stdout:5495 read 0x97162ae thru 0x9726267 (0xffba bytes) 2024-04-24T18:09:01.164 INFO:teuthology.orchestra.run.smithi049.stdout:5496 trunc from 0xbed8e33 to 0x3ac8767 2024-04-24T18:09:01.164 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:01.172+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:01.176 INFO:teuthology.orchestra.run.smithi049.stdout:5497 punch from 0x242cfc to 0x246868, (0x3b6c bytes) 2024-04-24T18:09:01.176 INFO:teuthology.orchestra.run.smithi049.stdout:5498 read 0xdec553 thru 0xdee3d1 (0x1e7f bytes) 2024-04-24T18:09:01.187 INFO:teuthology.orchestra.run.smithi049.stdout:5500 write 0x8b3fec7 thru 0x8b478b7 (0x79f1 bytes) 2024-04-24T18:09:01.199 INFO:teuthology.orchestra.run.smithi049.stdout:5503 read 0x10075f1 thru 0x10121d7 (0xabe7 bytes) 2024-04-24T18:09:01.200 INFO:teuthology.orchestra.run.smithi049.stdout:5504 read 0xd0741a thru 0xd122e6 (0xaecd bytes) 2024-04-24T18:09:01.200 INFO:teuthology.orchestra.run.smithi049.stdout:5506 read 0x5f70bce thru 0x5f7d33a (0xc76d bytes) 2024-04-24T18:09:01.200 INFO:teuthology.orchestra.run.smithi049.stdout:5507 read 0x2b5d06c thru 0x2b5eb45 (0x1ada bytes) 2024-04-24T18:09:01.201 INFO:teuthology.orchestra.run.smithi049.stdout:5508 punch from 0x3164dae to 0x316e516, (0x9768 bytes) 2024-04-24T18:09:01.212 INFO:teuthology.orchestra.run.smithi049.stdout:5509 writesame 0xdf54214 thru 0xdf6137e data_size 0x1(0xd16b bytes) 2024-04-24T18:09:01.212 INFO:teuthology.orchestra.run.smithi049.stdout:5510 write 0x8cd46 thru 0x95937 (0x8bf2 bytes) 2024-04-24T18:09:01.223 INFO:teuthology.orchestra.run.smithi049.stdout:5511 trunc from 0xdf6137f to 0x7a0ce79 2024-04-24T18:09:01.239 INFO:teuthology.orchestra.run.smithi049.stdout:5514 punch from 0x98298c to 0x98bf44, (0x95b8 bytes) 2024-04-24T18:09:01.239 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.239 INFO:teuthology.orchestra.run.smithi049.stdout:5518 read 0x2d25933 thru 0x2d2ecd4 (0x93a2 bytes) 2024-04-24T18:09:01.246 INFO:teuthology.orchestra.run.smithi049.stdout:5519 writesame 0x92a35ed thru 0x92a733c data_size 0x30(0x3d50 bytes) 2024-04-24T18:09:01.249 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:01.249 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:01.250 INFO:teuthology.orchestra.run.smithi049.stdout:5520 trunc from 0x92a733d to 0x616f13c 2024-04-24T18:09:01.261 INFO:teuthology.orchestra.run.smithi049.stdout:5521 write 0x628ad07 thru 0x6298035 (0xd32f bytes) 2024-04-24T18:09:01.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:01.266 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:01.273+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:01.283 INFO:teuthology.orchestra.run.smithi049.stdout:5522 trunc from 0x6298036 to 0xb8f6e9c 2024-04-24T18:09:01.289 INFO:teuthology.orchestra.run.smithi049.stdout:5523 read 0x19c4564 thru 0x19cc49c (0x7f39 bytes) 2024-04-24T18:09:01.289 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.289 INFO:teuthology.orchestra.run.smithi049.stdout:5525 punch from 0x8ce70f6 to 0x8cf1e64, (0xad6e bytes) 2024-04-24T18:09:01.289 INFO:teuthology.orchestra.run.smithi049.stdout:5526 write 0x4136072 thru 0x413d80f (0x779e bytes) 2024-04-24T18:09:01.301 INFO:teuthology.orchestra.run.smithi049.stdout:5527 write 0x1959b38 thru 0x1967975 (0xde3e bytes) 2024-04-24T18:09:01.327 INFO:teuthology.orchestra.run.smithi049.stdout:5528 trunc from 0xb8f6e9c to 0x264d3f9 2024-04-24T18:09:01.357 INFO:teuthology.orchestra.run.smithi049.stdout:5529 write 0x631bb8b thru 0x63230dc (0x7552 bytes) 2024-04-24T18:09:01.367 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:01.375+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:01.370 INFO:teuthology.orchestra.run.smithi049.stdout:5530 punch from 0x20cb694 to 0x20db067, (0xf9d3 bytes) 2024-04-24T18:09:01.371 INFO:teuthology.orchestra.run.smithi049.stdout:5532 trunc from 0x63230dd to 0x77d7dbc 2024-04-24T18:09:01.378 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.378 INFO:teuthology.orchestra.run.smithi049.stdout:5534 write 0x66794f4 thru 0x6684109 (0xac16 bytes) 2024-04-24T18:09:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:5535 read 0x2ed2468 thru 0x2eddf63 (0xbafc bytes) 2024-04-24T18:09:01.384 INFO:teuthology.orchestra.run.smithi049.stdout:5536 punch from 0x1b8b908 to 0x1b9a79f, (0xee97 bytes) 2024-04-24T18:09:01.385 INFO:teuthology.orchestra.run.smithi049.stdout:5537 write 0x52ba881 thru 0x52c80a4 (0xd824 bytes) 2024-04-24T18:09:01.400 INFO:teuthology.orchestra.run.smithi049.stdout:5538 read 0x4c89622 thru 0x4c98cc5 (0xf6a4 bytes) 2024-04-24T18:09:01.402 INFO:teuthology.orchestra.run.smithi049.stdout:5539 trunc from 0x77d7dbc to 0x5b0e30b 2024-04-24T18:09:01.415 INFO:teuthology.orchestra.run.smithi049.stdout:5541 writesame 0x25255a thru 0x25c64c data_size 0x1(0xa0f3 bytes) 2024-04-24T18:09:01.426 INFO:teuthology.orchestra.run.smithi049.stdout:5542 punch from 0x173b76b to 0x1748c3d, (0xd4d2 bytes) 2024-04-24T18:09:01.427 INFO:teuthology.orchestra.run.smithi049.stdout:5544 read 0x48b6ef5 thru 0x48c4024 (0xd130 bytes) 2024-04-24T18:09:01.442 INFO:teuthology.orchestra.run.smithi049.stdout:5545 writesame 0xe50cbd4 thru 0xe51483f data_size 0x2(0x7c6c bytes) 2024-04-24T18:09:01.442 INFO:teuthology.orchestra.run.smithi049.stdout:5546 read 0x6adc290 thru 0x6ae6f5b (0xaccc bytes) 2024-04-24T18:09:01.442 INFO:teuthology.orchestra.run.smithi049.stdout:5548 write 0x34dc105 thru 0x34ea6d0 (0xe5cc bytes) 2024-04-24T18:09:01.445 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:01.446 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:01.456 INFO:teuthology.orchestra.run.smithi049.stdout:5549 read 0xe5028a thru 0xe5136e (0x10e5 bytes) 2024-04-24T18:09:01.456 INFO:teuthology.orchestra.run.smithi049.stdout:5550 read 0xbb30f6 thru 0xbbf6d9 (0xc5e4 bytes) 2024-04-24T18:09:01.459 INFO:teuthology.orchestra.run.smithi049.stdout:5552 trunc from 0xe514840 to 0xc2ca3a0 2024-04-24T18:09:01.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:01.468 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:01.476+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:01.470 INFO:teuthology.orchestra.run.smithi049.stdout:5554 write 0xcba3130 thru 0xcbaeb3f (0xba10 bytes) 2024-04-24T18:09:01.484 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.484 INFO:teuthology.orchestra.run.smithi049.stdout:5556 write 0xc2d6490 thru 0xc2deabf (0x8630 bytes) 2024-04-24T18:09:01.496 INFO:teuthology.orchestra.run.smithi049.stdout:5557 write 0x1c8a332 thru 0x1c9042d (0x60fc bytes) 2024-04-24T18:09:01.499 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.499 INFO:teuthology.orchestra.run.smithi049.stdout:5559 read 0x2bfddb9 thru 0x2c07f62 (0xa1aa bytes) 2024-04-24T18:09:01.499 INFO:teuthology.orchestra.run.smithi049.stdout:5560 write 0x4439f89 thru 0x44480bc (0xe134 bytes) 2024-04-24T18:09:01.516 INFO:teuthology.orchestra.run.smithi049.stdout:5561 read 0xae0803a thru 0xae0a24a (0x2211 bytes) 2024-04-24T18:09:01.529 INFO:teuthology.orchestra.run.smithi049.stdout:5562 trunc from 0xcbaeb40 to 0x13534fc 2024-04-24T18:09:01.560 INFO:teuthology.orchestra.run.smithi049.stdout:5565 write 0xb562b7 thru 0xb62fe7 (0xcd31 bytes) 2024-04-24T18:09:01.568 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:01.576+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:01.580 INFO:teuthology.orchestra.run.smithi049.stdout:5566 trunc from 0x13534fc to 0x9ead873 2024-04-24T18:09:01.586 INFO:teuthology.orchestra.run.smithi049.stdout:5567 write 0x4a9f9ac thru 0x4aa6d3f (0x7394 bytes) 2024-04-24T18:09:01.601 INFO:teuthology.orchestra.run.smithi049.stdout:5568 write 0x836d633 thru 0x837be6c (0xe83a bytes) 2024-04-24T18:09:01.617 INFO:teuthology.orchestra.run.smithi049.stdout:5570 write 0x3c11670 thru 0x3c19937 (0x82c8 bytes) 2024-04-24T18:09:01.630 INFO:teuthology.orchestra.run.smithi049.stdout:5573 write 0x9e04281 thru 0x9e10933 (0xc6b3 bytes) 2024-04-24T18:09:01.643 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:01.643 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:01.645 INFO:teuthology.orchestra.run.smithi049.stdout:5574 punch from 0x52baac4 to 0x52c7ef2, (0xd42e bytes) 2024-04-24T18:09:01.645 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.645 INFO:teuthology.orchestra.run.smithi049.stdout:5580 read 0x5b28d98 thru 0x5b2ddea (0x5053 bytes) 2024-04-24T18:09:01.646 INFO:teuthology.orchestra.run.smithi049.stdout:5581 read 0x4ccaa03 thru 0x4cd02f9 (0x58f7 bytes) 2024-04-24T18:09:01.646 INFO:teuthology.orchestra.run.smithi049.stdout:5582 writesame 0x1eab030 thru 0x1eb41fb data_size 0x7c(0x91cc bytes) 2024-04-24T18:09:01.646 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:01.646 INFO:teuthology.orchestra.run.smithi049.stdout:{"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","health":{"status":"HEALTH_WARN","checks":{"OSDMAP_FLAGS":{"severity":"HEALTH_WARN","summary":{"message":"noscrub flag(s) set","count":7},"muted":false}},"mutes":[]},"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":114,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":103,"num_osds":8,"num_up_osds":8,"osd_up_since":1713982132,"num_in_osds":8,"osd_in_since":1713982075,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":40},{"state_name":"peering","count":5}],"num_pgs":45,"num_pools":4,"num_objects":3706,"data_bytes":1329177800,"bytes_used":2604040192,"bytes_avail":770490073088,"bytes_total":773094113280,"inactive_pgs_ratio":0.1111111119389534,"recovering_objects_per_sec":181,"recovering_bytes_per_sec":6460264,"recovering_keys_per_sec":17,"num_objects_recovered":726,"num_bytes_recovered":25845340,"num_keys_recovered":70,"read_bytes_sec":6810007,"write_bytes_sec":6479566,"read_op_per_sec":2655,"write_op_per_sec":95},"fsmap":{"epoch":1,"by_rank":[],"up:standby":0},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2024-04-24T18:08:19.309304+0000","services":{}},"progress_events":{}} 2024-04-24T18:09:01.660 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:01.661 INFO:teuthology.orchestra.run.smithi049.stdout:5584 read 0x51a1ebe thru 0x51a965f (0x77a2 bytes) 2024-04-24T18:09:01.661 INFO:teuthology.orchestra.run.smithi049.stdout:5585 punch from 0x6d45a71 to 0x6d49813, (0x3da2 bytes) 2024-04-24T18:09:01.661 INFO:teuthology.orchestra.run.smithi049.stdout:5586 punch from 0x3fbbec9 to 0x3fc1d25, (0x5e5c bytes) 2024-04-24T18:09:01.667 INFO:tasks.ceph.ceph_manager.ceph:making progress, resetting timeout 2024-04-24T18:09:01.667 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:09:01.668 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:01.677+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:01.672 INFO:teuthology.orchestra.run.smithi049.stdout:5587 write 0xdee52ca thru 0xdef03cb (0xb102 bytes) 2024-04-24T18:09:01.687 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.687 INFO:teuthology.orchestra.run.smithi049.stdout:5590 read 0x2c1f79e thru 0x2c228c6 (0x3129 bytes) 2024-04-24T18:09:01.687 INFO:teuthology.orchestra.run.smithi049.stdout:5591 read 0x47d6003 thru 0x47e417d (0xe17b bytes) 2024-04-24T18:09:01.688 INFO:teuthology.orchestra.run.smithi049.stdout:5593 write 0xbd2f8a6 thru 0xbd30bf6 (0x1351 bytes) 2024-04-24T18:09:01.693 INFO:teuthology.orchestra.run.smithi049.stdout:5594 write 0x305d9e4 thru 0x306a81f (0xce3c bytes) 2024-04-24T18:09:01.731 INFO:teuthology.orchestra.run.smithi049.stdout:5596 trunc from 0xdef03cc to 0xd1b927 2024-04-24T18:09:01.770 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:01.778+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:01.776 INFO:teuthology.orchestra.run.smithi049.stdout:5601 read 0xaa33b3 thru 0xaa4c4a (0x1898 bytes) 2024-04-24T18:09:01.785 INFO:teuthology.orchestra.run.smithi049.stdout:5602 write 0x43ba256 thru 0x43bf066 (0x4e11 bytes) 2024-04-24T18:09:01.795 INFO:teuthology.orchestra.run.smithi049.stdout:5603 punch from 0xd164bf to 0xd1fb77, (0x96b8 bytes) 2024-04-24T18:09:01.795 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:01.795 INFO:teuthology.orchestra.run.smithi049.stdout:5606 read 0xedee27 thru 0xee0d04 (0x1ede bytes) 2024-04-24T18:09:01.811 INFO:teuthology.orchestra.run.smithi049.stdout:5608 writesame 0xe1e0c69 thru 0xe1e9cce data_size 0x7a(0x9066 bytes) 2024-04-24T18:09:01.812 INFO:teuthology.orchestra.run.smithi049.stdout:5609 punch from 0x900a634 to 0x901a4fe, (0xfeca bytes) 2024-04-24T18:09:01.812 INFO:teuthology.orchestra.run.smithi049.stdout:5610 read 0xd97d63d thru 0xd98c1bf (0xeb83 bytes) 2024-04-24T18:09:01.812 INFO:teuthology.orchestra.run.smithi049.stdout:5611 write 0xc6fada9 thru 0xc7072e9 (0xc541 bytes) 2024-04-24T18:09:01.826 INFO:teuthology.orchestra.run.smithi049.stdout:5612 writesame 0xbe3536e thru 0xbe3e0b8 data_size 0x3(0x8d4b bytes) 2024-04-24T18:09:01.840 INFO:teuthology.orchestra.run.smithi049.stdout:5613 write 0x16de23e thru 0x16ec41a (0xe1dd bytes) 2024-04-24T18:09:01.844 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:01.845 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:01.855 INFO:teuthology.orchestra.run.smithi049.stdout:5616 writesame 0x529c47a thru 0x529d947 data_size 0x1(0x14ce bytes) 2024-04-24T18:09:01.860 INFO:teuthology.orchestra.run.smithi049.stdout:5617 punch from 0xc745a9 to 0xc7e2ea, (0x9d41 bytes) 2024-04-24T18:09:01.860 INFO:teuthology.orchestra.run.smithi049.stdout:5618 read 0x43f11bf thru 0x43fa2a5 (0x90e7 bytes) 2024-04-24T18:09:01.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:01.861 INFO:teuthology.orchestra.run.smithi049.stdout:5619 read 0x41b5e5a thru 0x41bb776 (0x591d bytes) 2024-04-24T18:09:01.861 INFO:teuthology.orchestra.run.smithi049.stdout:5620 read 0xc024c36 thru 0xc033bd6 (0xefa1 bytes) 2024-04-24T18:09:01.862 INFO:teuthology.orchestra.run.smithi049.stdout:5621 read 0x7966e4f thru 0x79694cc (0x267e bytes) 2024-04-24T18:09:01.862 INFO:teuthology.orchestra.run.smithi049.stdout:5623 read 0x13ca7d3 thru 0x13cddc3 (0x35f1 bytes) 2024-04-24T18:09:01.862 INFO:teuthology.orchestra.run.smithi049.stdout:5624 trunc from 0xe1e9ccf to 0xd079d90 2024-04-24T18:09:01.870 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:01.879+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:01.873 INFO:teuthology.orchestra.run.smithi049.stdout:5625 punch from 0x8715754 to 0x8718fc0, (0x386c bytes) 2024-04-24T18:09:01.873 INFO:teuthology.orchestra.run.smithi049.stdout:5626 read 0x3adca6b thru 0x3ae53a1 (0x8937 bytes) 2024-04-24T18:09:01.873 INFO:teuthology.orchestra.run.smithi049.stdout:5627 write 0xa42fad2 thru 0xa438141 (0x8670 bytes) 2024-04-24T18:09:01.888 INFO:teuthology.orchestra.run.smithi049.stdout:5628 write 0x915cc65 thru 0x916c398 (0xf734 bytes) 2024-04-24T18:09:01.903 INFO:teuthology.orchestra.run.smithi049.stdout:5629 punch from 0x5361ce3 to 0x5366eb4, (0x51d1 bytes) 2024-04-24T18:09:01.903 INFO:teuthology.orchestra.run.smithi049.stdout:5630 writesame 0x64b31c5 thru 0x64baa80 data_size 0x2(0x78bc bytes) 2024-04-24T18:09:01.926 INFO:teuthology.orchestra.run.smithi049.stdout:5631 write 0xd8ca4c0 thru 0xd8cbb84 (0x16c5 bytes) 2024-04-24T18:09:01.930 INFO:teuthology.orchestra.run.smithi049.stdout:5632 writesame 0x966e8f0 thru 0x96756aa data_size 0x1(0x6dbb bytes) 2024-04-24T18:09:01.935 INFO:teuthology.orchestra.run.smithi049.stdout:5633 write 0x9328fee thru 0x932b421 (0x2434 bytes) 2024-04-24T18:09:01.953 INFO:teuthology.orchestra.run.smithi049.stdout:5634 trunc from 0xd8cbb85 to 0x53ed8f 2024-04-24T18:09:01.971 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:01.979+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:01.998 INFO:teuthology.orchestra.run.smithi049.stdout:5635 read 0x4252bb thru 0x429d29 (0x4a6f bytes) 2024-04-24T18:09:02.006 INFO:teuthology.orchestra.run.smithi049.stdout:5638 trunc from 0x53ed8f to 0x7ef3d80 2024-04-24T18:09:02.016 INFO:teuthology.orchestra.run.smithi049.stdout:5639 trunc from 0x7ef3d80 to 0xb717b63 2024-04-24T18:09:02.022 INFO:teuthology.orchestra.run.smithi049.stdout:5640 read 0x9ba28f3 thru 0x9ba7a7e (0x518c bytes) 2024-04-24T18:09:02.022 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:02.022 INFO:teuthology.orchestra.run.smithi049.stdout:5642 read 0x18a2176 thru 0x18a94ab (0x7336 bytes) 2024-04-24T18:09:02.022 INFO:teuthology.orchestra.run.smithi049.stdout:5643 punch from 0x2ee1b9c to 0x2eec07a, (0xa4de bytes) 2024-04-24T18:09:02.022 INFO:teuthology.orchestra.run.smithi049.stdout:5644 punch from 0x2c5a4e6 to 0x2c68535, (0xe04f bytes) 2024-04-24T18:09:02.022 INFO:teuthology.orchestra.run.smithi049.stdout:5645 read 0x9c1289c thru 0x9c1f4d7 (0xcc3c bytes) 2024-04-24T18:09:02.023 INFO:teuthology.orchestra.run.smithi049.stdout:5646 write 0x11b7454 thru 0x11b87ed (0x139a bytes) 2024-04-24T18:09:02.027 INFO:teuthology.orchestra.run.smithi049.stdout:5647 trunc from 0xb717b63 to 0xa59297c 2024-04-24T18:09:02.031 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:02.031 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:02.041 INFO:teuthology.orchestra.run.smithi049.stdout:5648 trunc from 0xa59297c to 0x37498f3 2024-04-24T18:09:02.047 INFO:teuthology.orchestra.run.smithi049.stdout:5649 punch from 0x349c623 to 0x34a9a61, (0xd43e bytes) 2024-04-24T18:09:02.048 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:02.048 INFO:teuthology.orchestra.run.smithi049.stdout:5652 punch from 0x235fd5 to 0x2417c2, (0xb7ed bytes) 2024-04-24T18:09:02.048 INFO:teuthology.orchestra.run.smithi049.stdout:5653 read 0x22001af thru 0x220aa55 (0xa8a7 bytes) 2024-04-24T18:09:02.048 INFO:teuthology.orchestra.run.smithi049.stdout:5655 read 0x37474d0 thru 0x37498f2 (0x2423 bytes) 2024-04-24T18:09:02.048 INFO:teuthology.orchestra.run.smithi049.stdout:5656 read 0x372533e thru 0x372ad31 (0x59f4 bytes) 2024-04-24T18:09:02.048 INFO:teuthology.orchestra.run.smithi049.stdout:5660 read 0x1e92b07 thru 0x1e950d1 (0x25cb bytes) 2024-04-24T18:09:02.048 INFO:teuthology.orchestra.run.smithi049.stdout:5662 read 0xafbec0 thru 0xb03d52 (0x7e93 bytes) 2024-04-24T18:09:02.049 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:02.056 INFO:teuthology.orchestra.run.smithi049.stdout:5663 write 0x5f25987 thru 0x5f28176 (0x27f0 bytes) 2024-04-24T18:09:02.065 INFO:teuthology.orchestra.run.smithi049.stdout:5665 read 0x447f023 thru 0x448b969 (0xc947 bytes) 2024-04-24T18:09:02.066 INFO:teuthology.orchestra.run.smithi049.stdout:5667 trunc from 0x5f28177 to 0x485c4b8 2024-04-24T18:09:02.071 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:02.079+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:5668 read 0x399f83f thru 0x39ab227 (0xb9e9 bytes) 2024-04-24T18:09:02.079 INFO:teuthology.orchestra.run.smithi049.stdout:5669 write 0x2d791fa thru 0x2d80b14 (0x791b bytes) 2024-04-24T18:09:02.104 INFO:teuthology.orchestra.run.smithi049.stdout:5670 write 0x921be87 thru 0x9221adc (0x5c56 bytes) 2024-04-24T18:09:02.115 INFO:teuthology.orchestra.run.smithi049.stdout:5671 write 0x58dcd77 thru 0x58e918a (0xc414 bytes) 2024-04-24T18:09:02.133 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:02.135 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:02.136 INFO:teuthology.orchestra.run.smithi049.stdout:5672 write 0x8f76db1 thru 0x8f7fd1c (0x8f6c bytes) 2024-04-24T18:09:02.148 INFO:teuthology.orchestra.run.smithi049.stdout:5673 write 0xd02914 thru 0xd0fca5 (0xd392 bytes) 2024-04-24T18:09:02.151 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-04-24T18:09:01.316784+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1329177800,"num_objects":3706,"num_object_clones":673,"num_object_copies":7412,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3704,"num_whiteouts":2330,"num_read":227105,"num_read_kb":733141,"num_write":21007,"num_write_kb":473743,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18656,"ondisk_log_size":18656,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2543008,"kb_used_data":76840,"kb_used_omap":43,"kb_used_meta":2466004,"kb_avail":752431712,"statfs":{"total":773094113280,"available":770490073088,"internally_reserved":0,"allocated":78684160,"data_stored":662705684,"data_compressed":12970785,"data_compressed_allocated":42500096,"data_compressed_original":634904840,"omap_allocated":44665,"internal_metadata":2525188487},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5582795,"num_objects":108,"num_object_clones":4,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":-432,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":101,"num_read":10428,"num_read_kb":26078,"num_write":352,"num_write_kb":25200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":402,"num_bytes_recovered":11389499,"num_keys_recovered":34,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000555"},"pg_stats":[{"pgid":"4.1f","version":"97'457","reported_seq":5382,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.901318+0000","last_change":"2024-04-24T18:08:55.899667+0000","last_active":"2024-04-24T18:08:55.899445+0000","last_peered":"2024-04-24T18:08:55.671181+0000","last_clean":"2024-04-24T18:08:18.970509+0000","last_became_active":"2024-04-24T18:08:54.974383+0000","last_became_peered":"2024-04-24T18:08:54.974383+0000","last_unstale":"2024-04-24T18:08:55.901318+0000","last_undegraded":"2024-04-24T18:08:55.901318+0000","last_fullsized":"2024-04-24T18:08:55.901318+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":457,"log_dups_size":0,"ondisk_log_size":457,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:39:05.236590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":48185911,"num_objects":139,"num_object_clones":25,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":94,"num_read":5428,"num_read_kb":27520,"num_write":459,"num_write_kb":11775,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[5],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"94'1135","reported_seq":5715,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.895044+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:08:55.895044+0000","last_peered":"2024-04-24T18:08:55.895044+0000","last_clean":"2024-04-24T18:08:55.895044+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:08:55.895044+0000","last_undegraded":"2024-04-24T18:08:55.895044+0000","last_fullsized":"2024-04-24T18:08:55.895044+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1135,"log_dups_size":0,"ondisk_log_size":1135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":38392536,"num_objects":139,"num_object_clones":24,"num_object_copies":278,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":139,"num_whiteouts":89,"num_read":9052,"num_read_kb":19486,"num_write":1680,"num_write_kb":8429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"99'512","reported_seq":3960,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.935539+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:08:56.935539+0000","last_peered":"2024-04-24T18:08:56.935539+0000","last_clean":"2024-04-24T18:08:56.935539+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:08:56.935539+0000","last_undegraded":"2024-04-24T18:08:56.935539+0000","last_fullsized":"2024-04-24T18:08:56.935539+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":32404686,"num_objects":138,"num_object_clones":25,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":89,"num_read":4444,"num_read_kb":14922,"num_write":490,"num_write_kb":11863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"97'791","reported_seq":7936,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.895918+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:08:55.895918+0000","last_peered":"2024-04-24T18:08:55.895918+0000","last_clean":"2024-04-24T18:08:55.895918+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:08:55.895918+0000","last_undegraded":"2024-04-24T18:08:55.895918+0000","last_fullsized":"2024-04-24T18:08:55.895918+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":791,"log_dups_size":0,"ondisk_log_size":791,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":39279832,"num_objects":108,"num_object_clones":23,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":63,"num_read":9978,"num_read_kb":21879,"num_write":1068,"num_write_kb":8279,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"97'384","reported_seq":7066,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.927203+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:08:56.927203+0000","last_peered":"2024-04-24T18:08:56.927203+0000","last_clean":"2024-04-24T18:08:56.927203+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:08:56.927203+0000","last_undegraded":"2024-04-24T18:08:56.927203+0000","last_fullsized":"2024-04-24T18:08:56.927203+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":384,"log_dups_size":0,"ondisk_log_size":384,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":45749737,"num_objects":112,"num_object_clones":24,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":61,"num_read":6776,"num_read_kb":30196,"num_write":366,"num_write_kb":18770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"94'450","reported_seq":10342,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.919230+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:08:56.919230+0000","last_peered":"2024-04-24T18:08:56.919230+0000","last_clean":"2024-04-24T18:08:56.919230+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:08:56.919230+0000","last_undegraded":"2024-04-24T18:08:56.919230+0000","last_fullsized":"2024-04-24T18:08:56.919230+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":450,"log_dups_size":0,"ondisk_log_size":450,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":41098372,"num_objects":120,"num_object_clones":27,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":68,"num_read":11189,"num_read_kb":27289,"num_write":394,"num_write_kb":11539,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"97'545","reported_seq":8037,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900063+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:08:55.900063+0000","last_peered":"2024-04-24T18:08:55.900063+0000","last_clean":"2024-04-24T18:08:55.900063+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:08:55.900063+0000","last_undegraded":"2024-04-24T18:08:55.900063+0000","last_fullsized":"2024-04-24T18:08:55.900063+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":545,"log_dups_size":0,"ondisk_log_size":545,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":39665781,"num_objects":116,"num_object_clones":19,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":77,"num_read":8848,"num_read_kb":27076,"num_write":565,"num_write_kb":13542,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"98'415","reported_seq":8350,"reported_epoch":99,"state":"peering","last_fresh":"2024-04-24T18:08:56.896474+0000","last_change":"2024-04-24T18:08:56.896899+0000","last_active":"2024-04-24T18:08:56.896439+0000","last_peered":"2024-04-24T18:08:56.041651+0000","last_clean":"2024-04-24T18:08:20.733439+0000","last_became_active":"2024-04-24T18:08:54.975461+0000","last_became_peered":"2024-04-24T18:08:54.975461+0000","last_unstale":"2024-04-24T18:08:56.896474+0000","last_undegraded":"2024-04-24T18:08:56.896474+0000","last_fullsized":"2024-04-24T18:08:56.896474+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":415,"log_dups_size":0,"ondisk_log_size":415,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:02:54.955318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":44496065,"num_objects":103,"num_object_clones":18,"num_object_copies":206,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":103,"num_whiteouts":60,"num_read":8906,"num_read_kb":26427,"num_write":406,"num_write_kb":26815,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"97'414","reported_seq":4852,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.002567+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:08:56.002567+0000","last_peered":"2024-04-24T18:08:56.002567+0000","last_clean":"2024-04-24T18:08:56.002567+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:08:56.002567+0000","last_undegraded":"2024-04-24T18:08:56.002567+0000","last_fullsized":"2024-04-24T18:08:56.002567+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":414,"log_dups_size":0,"ondisk_log_size":414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":41936818,"num_objects":114,"num_object_clones":26,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":72,"num_read":5026,"num_read_kb":19857,"num_write":381,"num_write_kb":15303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"97'1035","reported_seq":3056,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.008253+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:08:56.008253+0000","last_peered":"2024-04-24T18:08:56.008253+0000","last_clean":"2024-04-24T18:08:56.008253+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:08:56.008253+0000","last_undegraded":"2024-04-24T18:08:56.008253+0000","last_fullsized":"2024-04-24T18:08:56.008253+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1035,"log_dups_size":0,"ondisk_log_size":1035,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":41063598,"num_objects":122,"num_object_clones":19,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":75,"num_read":6837,"num_read_kb":20698,"num_write":1522,"num_write_kb":17149,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-24T18:08:54.963507+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:08:54.963507+0000","last_peered":"2024-04-24T18:08:54.963507+0000","last_clean":"2024-04-24T18:08:54.963507+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:08:54.963507+0000","last_undegraded":"2024-04-24T18:08:54.963507+0000","last_fullsized":"2024-04-24T18:08:54.963507+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":64,"reported_epoch":97,"state":"active+clean","last_fresh":"2024-04-24T18:08:54.963544+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:08:54.963544+0000","last_peered":"2024-04-24T18:08:54.963544+0000","last_clean":"2024-04-24T18:08:54.963544+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:08:54.963544+0000","last_undegraded":"2024-04-24T18:08:54.963544+0000","last_fullsized":"2024-04-24T18:08:54.963544+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"94'499","reported_seq":4408,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.012311+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:08:56.012311+0000","last_peered":"2024-04-24T18:08:56.012311+0000","last_clean":"2024-04-24T18:08:56.012311+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:08:56.012311+0000","last_undegraded":"2024-04-24T18:08:56.012311+0000","last_fullsized":"2024-04-24T18:08:56.012311+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":59713549,"num_objects":122,"num_object_clones":24,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":74,"num_read":4646,"num_read_kb":29075,"num_write":538,"num_write_kb":18441,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":88,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898146+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:55.898146+0000","last_peered":"2024-04-24T18:08:55.898146+0000","last_clean":"2024-04-24T18:08:55.898146+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:55.898146+0000","last_undegraded":"2024-04-24T18:08:55.898146+0000","last_fullsized":"2024-04-24T18:08:55.898146+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898226+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:55.898226+0000","last_peered":"2024-04-24T18:08:55.898226+0000","last_clean":"2024-04-24T18:08:55.898226+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:55.898226+0000","last_undegraded":"2024-04-24T18:08:55.898226+0000","last_fullsized":"2024-04-24T18:08:55.898226+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"99'822","reported_seq":4432,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894848+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:08:56.894848+0000","last_peered":"2024-04-24T18:08:56.894848+0000","last_clean":"2024-04-24T18:08:56.894848+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:08:56.894848+0000","last_undegraded":"2024-04-24T18:08:56.894848+0000","last_fullsized":"2024-04-24T18:08:56.894848+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":822,"log_dups_size":0,"ondisk_log_size":822,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":29722279,"num_objects":114,"num_object_clones":18,"num_object_copies":228,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":114,"num_whiteouts":82,"num_read":6845,"num_read_kb":12351,"num_write":1025,"num_write_kb":15481,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"61'1","reported_seq":89,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900310+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:55.900310+0000","last_peered":"2024-04-24T18:08:55.900310+0000","last_clean":"2024-04-24T18:08:55.900310+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:55.900310+0000","last_undegraded":"2024-04-24T18:08:55.900310+0000","last_fullsized":"2024-04-24T18:08:55.900310+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:06.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":90,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191347+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:56.191347+0000","last_peered":"2024-04-24T18:08:56.191347+0000","last_clean":"2024-04-24T18:08:56.191347+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:56.191347+0000","last_undegraded":"2024-04-24T18:08:56.191347+0000","last_fullsized":"2024-04-24T18:08:56.191347+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"97'733","reported_seq":6276,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.904960+0000","last_change":"2024-04-24T18:08:55.901049+0000","last_active":"2024-04-24T18:08:55.900833+0000","last_peered":"2024-04-24T18:08:55.613013+0000","last_clean":"2024-04-24T18:08:19.722254+0000","last_became_active":"2024-04-24T18:08:54.974371+0000","last_became_peered":"2024-04-24T18:08:54.974371+0000","last_unstale":"2024-04-24T18:08:55.904960+0000","last_undegraded":"2024-04-24T18:08:55.904960+0000","last_fullsized":"2024-04-24T18:08:55.904960+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":733,"log_dups_size":0,"ondisk_log_size":733,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:44:09.259454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":53469738,"num_objects":138,"num_object_clones":21,"num_object_copies":276,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":138,"num_whiteouts":88,"num_read":8268,"num_read_kb":36769,"num_write":822,"num_write_kb":29777,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[4],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":62,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.893935+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:55.893935+0000","last_peered":"2024-04-24T18:08:55.893935+0000","last_clean":"2024-04-24T18:08:55.893935+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:55.893935+0000","last_undegraded":"2024-04-24T18:08:55.893935+0000","last_fullsized":"2024-04-24T18:08:55.893935+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":90,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191290+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:56.191290+0000","last_peered":"2024-04-24T18:08:56.191290+0000","last_clean":"2024-04-24T18:08:56.191290+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:56.191290+0000","last_undegraded":"2024-04-24T18:08:56.191290+0000","last_fullsized":"2024-04-24T18:08:56.191290+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"94'372","reported_seq":4298,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.992256+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:08:55.992256+0000","last_peered":"2024-04-24T18:08:55.992256+0000","last_clean":"2024-04-24T18:08:55.992256+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:08:55.992256+0000","last_undegraded":"2024-04-24T18:08:55.992256+0000","last_fullsized":"2024-04-24T18:08:55.992256+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":372,"log_dups_size":0,"ondisk_log_size":372,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":30102454,"num_objects":113,"num_object_clones":28,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":66,"num_read":5535,"num_read_kb":23916,"num_write":471,"num_write_kb":12527,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'1","reported_seq":89,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.894711+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:55.894711+0000","last_peered":"2024-04-24T18:08:55.894711+0000","last_clean":"2024-04-24T18:08:55.894711+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:55.894711+0000","last_undegraded":"2024-04-24T18:08:55.894711+0000","last_fullsized":"2024-04-24T18:08:55.894711+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":288,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.191354+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:56.191354+0000","last_peered":"2024-04-24T18:08:56.191354+0000","last_clean":"2024-04-24T18:08:56.191354+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:56.191354+0000","last_undegraded":"2024-04-24T18:08:56.191354+0000","last_fullsized":"2024-04-24T18:08:56.191354+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":77,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.894092+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:08:55.894092+0000","last_peered":"2024-04-24T18:08:55.894092+0000","last_clean":"2024-04-24T18:08:55.894092+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:08:55.894092+0000","last_undegraded":"2024-04-24T18:08:55.894092+0000","last_fullsized":"2024-04-24T18:08:55.894092+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"97'764","reported_seq":8211,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.008859+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:08:56.008859+0000","last_peered":"2024-04-24T18:08:56.008859+0000","last_clean":"2024-04-24T18:08:56.008859+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:08:56.008859+0000","last_undegraded":"2024-04-24T18:08:56.008859+0000","last_fullsized":"2024-04-24T18:08:56.008859+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":764,"log_dups_size":0,"ondisk_log_size":764,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":43062717,"num_objects":98,"num_object_clones":16,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":58,"num_read":11080,"num_read_kb":29375,"num_write":966,"num_write_kb":21026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"97'494","reported_seq":5479,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898225+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:08:55.898225+0000","last_peered":"2024-04-24T18:08:55.898225+0000","last_clean":"2024-04-24T18:08:55.898225+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:08:55.898225+0000","last_undegraded":"2024-04-24T18:08:55.898225+0000","last_fullsized":"2024-04-24T18:08:55.898225+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":494,"log_dups_size":0,"ondisk_log_size":494,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":48776479,"num_objects":116,"num_object_clones":18,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":83,"num_read":6300,"num_read_kb":29404,"num_write":416,"num_write_kb":23341,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.899385+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:55.899385+0000","last_peered":"2024-04-24T18:08:55.899385+0000","last_clean":"2024-04-24T18:08:55.899385+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:55.899385+0000","last_undegraded":"2024-04-24T18:08:55.899385+0000","last_fullsized":"2024-04-24T18:08:55.899385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"99'535","reported_seq":4575,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.958671+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:08:56.958671+0000","last_peered":"2024-04-24T18:08:56.958671+0000","last_clean":"2024-04-24T18:08:56.958671+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:08:56.958671+0000","last_undegraded":"2024-04-24T18:08:56.958671+0000","last_fullsized":"2024-04-24T18:08:56.958671+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":40019230,"num_objects":107,"num_object_clones":19,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":62,"num_read":5850,"num_read_kb":14786,"num_write":559,"num_write_kb":7238,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":91,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.900081+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:55.900081+0000","last_peered":"2024-04-24T18:08:55.900081+0000","last_clean":"2024-04-24T18:08:55.900081+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:55.900081+0000","last_undegraded":"2024-04-24T18:08:55.900081+0000","last_fullsized":"2024-04-24T18:08:55.900081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"97'455","reported_seq":8070,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.917598+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:08:56.917598+0000","last_peered":"2024-04-24T18:08:56.917598+0000","last_clean":"2024-04-24T18:08:56.917598+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:08:56.917598+0000","last_undegraded":"2024-04-24T18:08:56.917598+0000","last_fullsized":"2024-04-24T18:08:56.917598+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":455,"log_dups_size":0,"ondisk_log_size":455,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":44322609,"num_objects":124,"num_object_clones":23,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":77,"num_read":8769,"num_read_kb":33538,"num_write":423,"num_write_kb":15372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"97'496","reported_seq":5791,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.898074+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:08:55.898074+0000","last_peered":"2024-04-24T18:08:55.898074+0000","last_clean":"2024-04-24T18:08:55.898074+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:08:55.898074+0000","last_undegraded":"2024-04-24T18:08:55.898074+0000","last_fullsized":"2024-04-24T18:08:55.898074+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":46300149,"num_objects":117,"num_object_clones":19,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":72,"num_read":6019,"num_read_kb":17989,"num_write":543,"num_write_kb":12639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"97'554","reported_seq":1979,"reported_epoch":98,"state":"peering","last_fresh":"2024-04-24T18:08:55.900408+0000","last_change":"2024-04-24T18:08:55.900231+0000","last_active":"2024-04-24T18:08:55.900030+0000","last_peered":"2024-04-24T18:08:55.671491+0000","last_clean":"2024-04-24T18:08:18.932949+0000","last_became_active":"2024-04-24T18:08:54.974371+0000","last_became_peered":"2024-04-24T18:08:54.974371+0000","last_unstale":"2024-04-24T18:08:55.900408+0000","last_undegraded":"2024-04-24T18:08:55.900408+0000","last_fullsized":"2024-04-24T18:08:55.900408+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":554,"log_dups_size":0,"ondisk_log_size":554,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:44:42.192974+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":23653650,"num_objects":84,"num_object_clones":7,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":59,"num_read":2223,"num_read_kb":7448,"num_write":638,"num_write_kb":10347,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[1],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"97'493","reported_seq":4184,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.000383+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:08:56.000383+0000","last_peered":"2024-04-24T18:08:56.000383+0000","last_clean":"2024-04-24T18:08:56.000383+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:08:56.000383+0000","last_undegraded":"2024-04-24T18:08:56.000383+0000","last_fullsized":"2024-04-24T18:08:56.000383+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":493,"log_dups_size":0,"ondisk_log_size":493,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":24426121,"num_objects":95,"num_object_clones":15,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":63,"num_read":6412,"num_read_kb":13446,"num_write":515,"num_write_kb":10233,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"97'440","reported_seq":5523,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.928037+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:08:56.928037+0000","last_peered":"2024-04-24T18:08:56.928037+0000","last_clean":"2024-04-24T18:08:56.928037+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:08:56.928037+0000","last_undegraded":"2024-04-24T18:08:56.928037+0000","last_fullsized":"2024-04-24T18:08:56.928037+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":440,"log_dups_size":0,"ondisk_log_size":440,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":39221197,"num_objects":94,"num_object_clones":15,"num_object_copies":188,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":94,"num_whiteouts":61,"num_read":6301,"num_read_kb":26476,"num_write":425,"num_write_kb":27931,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"94'468","reported_seq":6852,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.911816+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:08:56.911816+0000","last_peered":"2024-04-24T18:08:56.911816+0000","last_clean":"2024-04-24T18:08:56.911816+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:08:56.911816+0000","last_undegraded":"2024-04-24T18:08:56.911816+0000","last_fullsized":"2024-04-24T18:08:56.911816+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":468,"log_dups_size":0,"ondisk_log_size":468,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":44049617,"num_objects":132,"num_object_clones":24,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":84,"num_read":6868,"num_read_kb":16837,"num_write":387,"num_write_kb":5434,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"97'434","reported_seq":5590,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.938722+0000","last_change":"2024-04-24T18:08:56.920936+0000","last_active":"2024-04-24T18:08:56.938722+0000","last_peered":"2024-04-24T18:08:56.938722+0000","last_clean":"2024-04-24T18:08:56.938722+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:08:56.938722+0000","last_undegraded":"2024-04-24T18:08:56.938722+0000","last_fullsized":"2024-04-24T18:08:56.938722+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":434,"log_dups_size":0,"ondisk_log_size":434,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":42388138,"num_objects":110,"num_object_clones":24,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":63,"num_read":5566,"num_read_kb":14066,"num_write":415,"num_write_kb":12876,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"99'643","reported_seq":3498,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894417+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:08:56.894417+0000","last_peered":"2024-04-24T18:08:56.894417+0000","last_clean":"2024-04-24T18:08:56.894417+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:08:56.894417+0000","last_undegraded":"2024-04-24T18:08:56.894417+0000","last_fullsized":"2024-04-24T18:08:56.894417+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":643,"log_dups_size":0,"ondisk_log_size":643,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":32472669,"num_objects":113,"num_object_clones":15,"num_object_copies":226,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":113,"num_whiteouts":70,"num_read":5542,"num_read_kb":17226,"num_write":728,"num_write_kb":7460,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"94'462","reported_seq":5843,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:55.899349+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:08:55.899349+0000","last_peered":"2024-04-24T18:08:55.899349+0000","last_clean":"2024-04-24T18:08:55.899349+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:08:55.899349+0000","last_undegraded":"2024-04-24T18:08:55.899349+0000","last_fullsized":"2024-04-24T18:08:55.899349+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":462,"log_dups_size":0,"ondisk_log_size":462,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":28645466,"num_objects":125,"num_object_clones":24,"num_object_copies":250,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":125,"num_whiteouts":79,"num_read":6684,"num_read_kb":15620,"num_write":391,"num_write_kb":7897,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"97'584","reported_seq":7578,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.014510+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:08:56.014510+0000","last_peered":"2024-04-24T18:08:56.014510+0000","last_clean":"2024-04-24T18:08:56.014510+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:08:56.014510+0000","last_undegraded":"2024-04-24T18:08:56.014510+0000","last_fullsized":"2024-04-24T18:08:56.014510+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":584,"log_dups_size":0,"ondisk_log_size":584,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":92624525,"num_objects":129,"num_object_clones":26,"num_object_copies":258,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":129,"num_whiteouts":83,"num_read":8296,"num_read_kb":46021,"num_write":663,"num_write_kb":29350,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"94'502","reported_seq":5426,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.897040+0000","last_change":"2024-04-24T18:08:56.897040+0000","last_active":"2024-04-24T18:08:56.897040+0000","last_peered":"2024-04-24T18:08:56.897040+0000","last_clean":"2024-04-24T18:08:56.897040+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:08:56.897040+0000","last_undegraded":"2024-04-24T18:08:56.897040+0000","last_fullsized":"2024-04-24T18:08:56.897040+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":502,"log_dups_size":0,"ondisk_log_size":502,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":25564751,"num_objects":120,"num_object_clones":16,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":82,"num_read":9144,"num_read_kb":29322,"num_write":422,"num_write_kb":8077,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"97'840","reported_seq":7979,"reported_epoch":99,"state":"peering","last_fresh":"2024-04-24T18:08:56.898707+0000","last_change":"2024-04-24T18:08:56.899109+0000","last_active":"2024-04-24T18:08:56.898674+0000","last_peered":"2024-04-24T18:08:56.161342+0000","last_clean":"2024-04-24T18:08:20.733644+0000","last_became_active":"2024-04-24T18:08:54.984176+0000","last_became_peered":"2024-04-24T18:08:54.984176+0000","last_unstale":"2024-04-24T18:08:56.898707+0000","last_undegraded":"2024-04-24T18:08:56.898707+0000","last_fullsized":"2024-04-24T18:08:56.898707+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":840,"log_dups_size":0,"ondisk_log_size":840,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:07.099674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":43298546,"num_objects":110,"num_object_clones":26,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":67,"num_read":9613,"num_read_kb":21463,"num_write":1186,"num_write_kb":22201,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[6],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"97'454","reported_seq":6563,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.003799+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:08:56.003799+0000","last_peered":"2024-04-24T18:08:56.003799+0000","last_clean":"2024-04-24T18:08:56.003799+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:08:56.003799+0000","last_undegraded":"2024-04-24T18:08:56.003799+0000","last_fullsized":"2024-04-24T18:08:56.003799+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":454,"log_dups_size":0,"ondisk_log_size":454,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":48303822,"num_objects":95,"num_object_clones":20,"num_object_copies":190,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":95,"num_whiteouts":57,"num_read":8379,"num_read_kb":22105,"num_write":399,"num_write_kb":7523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"97'425","reported_seq":4884,"reported_epoch":98,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.013828+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:08:56.013828+0000","last_peered":"2024-04-24T18:08:56.013828+0000","last_clean":"2024-04-24T18:08:56.013828+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:08:56.013828+0000","last_undegraded":"2024-04-24T18:08:56.013828+0000","last_fullsized":"2024-04-24T18:08:56.013828+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":19979170,"num_objects":107,"num_object_clones":20,"num_object_copies":214,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":107,"num_whiteouts":69,"num_read":6414,"num_read_kb":11795,"num_write":358,"num_write_kb":6933,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"99'930","reported_seq":3744,"reported_epoch":99,"state":"active+clean","last_fresh":"2024-04-24T18:08:56.894424+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:08:56.894424+0000","last_peered":"2024-04-24T18:08:56.894424+0000","last_clean":"2024-04-24T18:08:56.894424+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:08:56.894424+0000","last_undegraded":"2024-04-24T18:08:56.894424+0000","last_fullsized":"2024-04-24T18:08:56.894424+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":930,"log_dups_size":0,"ondisk_log_size":930,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":56197220,"num_objects":128,"num_object_clones":25,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":81,"num_read":5751,"num_read_kb":28663,"num_write":1227,"num_write_kb":15759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1328587432,"num_objects":3702,"num_object_clones":673,"num_object_copies":7404,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3702,"num_whiteouts":2328,"num_read":226989,"num_read_kb":733041,"num_write":20848,"num_write_kb":471327,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":51,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":70193152,"data_stored":666267624,"data_compressed":13212412,"data_compressed_allocated":43180032,"data_compressed_original":642892040,"omap_allocated":9738,"internal_metadata":0},"log_size":18537,"ondisk_log_size":18537,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574871,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":317532,"kb_used_data":7868,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94054308,"statfs":{"total":96636764160,"available":96311611392,"internally_reserved":0,"allocated":8056832,"data_stored":68522011,"data_compressed":1018862,"data_compressed_allocated":4190208,"data_compressed_original":65758884,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.40799999999999997}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.375}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.437},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.32000000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.30099999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.33300000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.42599999999999999}]}]},{"osd":6,"up_from":95,"seq":408021893122,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310184,"kb_used_data":7452,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94061656,"statfs":{"total":96636764160,"available":96319135744,"internally_reserved":0,"allocated":7630848,"data_stored":62744965,"data_compressed":1452463,"data_compressed_allocated":4415488,"data_compressed_original":60366517,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.316}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42399999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39800000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.43099999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46600000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574870,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323224,"kb_used_data":13580,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94048616,"statfs":{"total":96636764160,"available":96305782784,"internally_reserved":0,"allocated":13905920,"data_stored":98128349,"data_compressed":1986573,"data_compressed_allocated":6615040,"data_compressed_original":92426435,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.15,"5min":14.15,"15min":14.15},"last":0.311}]},{"osd":2,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.36199999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":0.39400000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":14.089,"5min":14.089,"15min":14.089},"last":0.432},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.34000000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.55400000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.45500000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":14.051,"5min":14.051,"15min":14.051},"last":0.501},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.45000000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574871,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320280,"kb_used_data":6540,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94051560,"statfs":{"total":96636764160,"available":96308797440,"internally_reserved":0,"allocated":6696960,"data_stored":60726874,"data_compressed":1053017,"data_compressed_allocated":3735552,"data_compressed_original":58676108,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.29799999999999999},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":26.936,"5min":26.936,"15min":26.936},"last":0.36099999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.34399999999999997},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.27400000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":14.371,"5min":14.371,"15min":14.371},"last":0.39900000000000002},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.973000000000001,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.414,"5min":14.414,"15min":14.414},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.50900000000000001},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":14.452,"5min":14.452,"15min":14.452},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"last":0.47699999999999998},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":14.486000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.57499999999999996}]}]},{"osd":2,"up_from":13,"seq":55834574870,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313664,"kb_used_data":4020,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058176,"statfs":{"total":96636764160,"available":96315572224,"internally_reserved":0,"allocated":4116480,"data_stored":24654978,"data_compressed":524522,"data_compressed_allocated":1728512,"data_compressed_original":23053518,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.27800000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.23899999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.26900000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.37},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.248}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.48299999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.499}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.51600000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.70299999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574871,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":320680,"kb_used_data":15136,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94051160,"statfs":{"total":96636764160,"available":96308387840,"internally_reserved":0,"allocated":15499264,"data_stored":142743749,"data_compressed":2373991,"data_compressed_allocated":8704000,"data_compressed_original":137212312,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.371},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.46000000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.48599999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.53400000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.504}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.64200000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.62},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.71999999999999997}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":0.754},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.54700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574871,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313732,"kb_used_data":12276,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94058108,"statfs":{"total":96636764160,"available":96315502592,"internally_reserved":0,"allocated":12570624,"data_stored":121128547,"data_compressed":2722834,"data_compressed_allocated":7471104,"data_compressed_original":116902284,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.441}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.36799999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.59899999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":2.5019999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":1.373},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.46800000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.629}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.61699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574871,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323712,"kb_used_data":9968,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94048128,"statfs":{"total":96636764160,"available":96305283072,"internally_reserved":0,"allocated":10207232,"data_stored":84056211,"data_compressed":1838523,"data_compressed_allocated":5640192,"data_compressed_original":80508782,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.39000000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.437}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.33600000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.42599999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.41199999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.45400000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.47299999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5607424,"data_stored":61064277,"data_compressed":1086854,"data_compressed_allocated":3821568,"data_compressed_original":59593612,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12914688,"data_stored":99716648,"data_compressed":2031230,"data_compressed_allocated":6791168,"data_compressed_original":94589123,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3043328,"data_stored":24998637,"data_compressed":561354,"data_compressed_allocated":1818624,"data_compressed_original":23971022,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14323712,"data_stored":142163648,"data_compressed":2373991,"data_compressed_allocated":8704000,"data_compressed_original":137212312,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11395072,"data_stored":120548446,"data_compressed":2722834,"data_compressed_allocated":7471104,"data_compressed_original":116902284,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9043968,"data_stored":83482366,"data_compressed":1838523,"data_compressed_allocated":5640192,"data_compressed_original":80508782,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7041024,"data_stored":66942060,"data_compressed":1587428,"data_compressed_allocated":4780032,"data_compressed_original":64929461,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6823936,"data_stored":67351542,"data_compressed":1010198,"data_compressed_allocated":4153344,"data_compressed_original":65185444,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:02.173 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:02.181+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:02.173 INFO:teuthology.orchestra.run.smithi049.stdout:5674 write 0xdaddb69 thru 0xdae99f8 (0xbe90 bytes) 2024-04-24T18:09:02.189 INFO:teuthology.orchestra.run.smithi049.stdout:5675 read 0xacf4849 thru 0xacfebb6 (0xa36e bytes) 2024-04-24T18:09:02.189 INFO:teuthology.orchestra.run.smithi049.stdout:5678 punch from 0xd3e936f to 0xd3f746d, (0xe0fe bytes) 2024-04-24T18:09:02.189 INFO:teuthology.orchestra.run.smithi049.stdout:5679 read 0x5f503b4 thru 0x5f5453a (0x4187 bytes) 2024-04-24T18:09:02.189 INFO:teuthology.orchestra.run.smithi049.stdout:5681 write 0x68bc885 thru 0x68c6074 (0x97f0 bytes) 2024-04-24T18:09:02.206 INFO:teuthology.orchestra.run.smithi049.stdout:5682 read 0x1024005 thru 0x102bad4 (0x7ad0 bytes) 2024-04-24T18:09:02.207 INFO:teuthology.orchestra.run.smithi049.stdout:5684 write 0x87da394 thru 0x87ea04a (0xfcb7 bytes) 2024-04-24T18:09:02.222 INFO:teuthology.orchestra.run.smithi049.stdout:5685 trunc from 0xdae99f9 to 0xd02abf5 2024-04-24T18:09:02.228 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:02.228 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:02.237 INFO:teuthology.orchestra.run.smithi049.stdout:5686 punch from 0x6352ff9 to 0x635422a, (0x1231 bytes) 2024-04-24T18:09:02.237 INFO:teuthology.orchestra.run.smithi049.stdout:5687 read 0x843238 thru 0x845c5e (0x2a27 bytes) 2024-04-24T18:09:02.237 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:02.238 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:02.239 INFO:teuthology.orchestra.run.smithi049.stdout:5689 trunc from 0xd02abf5 to 0xaba0fec 2024-04-24T18:09:02.245 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:02.249 INFO:teuthology.orchestra.run.smithi049.stdout:5691 trunc from 0xaba0fec to 0x74e4dba 2024-04-24T18:09:02.267 INFO:teuthology.orchestra.run.smithi049.stdout:5692 read 0x3ad6c07 thru 0x3ae369d (0xca97 bytes) 2024-04-24T18:09:02.267 INFO:teuthology.orchestra.run.smithi049.stdout:5694 read 0x33ebdb8 thru 0x33f9155 (0xd39e bytes) 2024-04-24T18:09:02.270 INFO:teuthology.orchestra.run.smithi049.stdout:5695 trunc from 0x74e4dba to 0x577b534 2024-04-24T18:09:02.274 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:02.282+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:02.294 INFO:teuthology.orchestra.run.smithi049.stdout:5696 write 0x73908f3 thru 0x739b29f (0xa9ad bytes) 2024-04-24T18:09:02.310 INFO:teuthology.orchestra.run.smithi049.stdout:5697 trunc from 0x739b2a0 to 0x827ce86 2024-04-24T18:09:02.318 INFO:teuthology.orchestra.run.smithi049.stdout:5699 trunc from 0x827ce86 to 0xa62ab63 2024-04-24T18:09:02.324 INFO:teuthology.orchestra.run.smithi049.stdout:5701 writesame 0x5ef7b6b thru 0x5efd308 data_size 0x5(0x579e bytes) 2024-04-24T18:09:02.335 INFO:teuthology.orchestra.run.smithi049.stdout:5702 read 0xb688a1 thru 0xb744bd (0xbc1d bytes) 2024-04-24T18:09:02.336 INFO:teuthology.orchestra.run.smithi049.stdout:5704 read 0x8b2767f thru 0x8b280fa (0xa7c bytes) 2024-04-24T18:09:02.336 INFO:teuthology.orchestra.run.smithi049.stdout:5705 read 0x613dfe1 thru 0x6143e51 (0x5e71 bytes) 2024-04-24T18:09:02.337 INFO:teuthology.orchestra.run.smithi049.stdout:5706 punch from 0x8c5a367 to 0x8c64f6a, (0xac03 bytes) 2024-04-24T18:09:02.337 INFO:teuthology.orchestra.run.smithi049.stdout:5707 clone 50 order 23 su 131072 sc 7 2024-04-24T18:09:02.375 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:02.384+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:02.390 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:02.391 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:02.401 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:02.485 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:02.493+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:02.570 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:02.570 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:02.576 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:02.584+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:02.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:02.593 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:02.677 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:02.686+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:02.752 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:02.752 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:02.763 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:02.767 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:02.778 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:02.786+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:02.787 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:02.807 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:02.827 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:02.846 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:02.865 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:02.879 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:02.888+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:02.884 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:02.894 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:02.894 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:02.894 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:02.895 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:02.902 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:02.904 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:02.905 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:02.905 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:02.931 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:09:02.980 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:02.989+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:03.081 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:03.089+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:03.083 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #48, image image_client.0-clone48 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent49 2024-04-24T18:09:03.084 INFO:teuthology.orchestra.run.smithi049.stdout:5708 punch from 0x5ad0dd3 to 0x5adc47e, (0xb6ab bytes) 2024-04-24T18:09:03.089 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:03.101 INFO:teuthology.orchestra.run.smithi049.stdout:5711 write 0xea91d19 thru 0xea93179 (0x1461 bytes) 2024-04-24T18:09:03.101 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:03.101 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:03.101 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:03.101 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:03.102 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:03.104 INFO:teuthology.orchestra.run.smithi049.stdout:5712 writesame 0xd4b9b3a thru 0xd4be291 data_size 0xc(0x4758 bytes) 2024-04-24T18:09:03.109 INFO:teuthology.orchestra.run.smithi049.stdout:5713 read 0x37ec018 thru 0x37f74e1 (0xb4ca bytes) 2024-04-24T18:09:03.109 INFO:teuthology.orchestra.run.smithi049.stdout:5715 punch from 0x7b310c1 to 0x7b3b7d9, (0xa718 bytes) 2024-04-24T18:09:03.109 INFO:teuthology.orchestra.run.smithi049.stdout:5718 writesame 0x85bcc4f thru 0x85bcc85 data_size 0x1(0x37 bytes) 2024-04-24T18:09:03.115 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:03.122 INFO:teuthology.orchestra.run.smithi049.stdout:5719 read 0x9ba95ea thru 0x9baea55 (0x546c bytes) 2024-04-24T18:09:03.122 INFO:teuthology.orchestra.run.smithi049.stdout:5720 writesame 0x6c8b5ac thru 0x6c99539 data_size 0x76(0xdf8e bytes) 2024-04-24T18:09:03.132 INFO:teuthology.orchestra.run.smithi049.stdout:5721 writesame 0x4eab5c5 thru 0x4eb922e data_size 0x2(0xdc6a bytes) 2024-04-24T18:09:03.154 INFO:teuthology.orchestra.run.smithi049.stdout:5723 write 0x2750f6f thru 0x27536a2 (0x2734 bytes) 2024-04-24T18:09:03.182 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:03.190+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:03.185 INFO:teuthology.orchestra.run.smithi049.stdout:5725 read 0x8486c76 thru 0x8487aec (0xe77 bytes) 2024-04-24T18:09:03.185 INFO:teuthology.orchestra.run.smithi049.stdout:5726 write 0x9ff3eb6 thru 0x9ffa2d9 (0x6424 bytes) 2024-04-24T18:09:03.187 INFO:teuthology.orchestra.run.smithi049.stdout:5727 read 0xe93cd8e thru 0xe9464db (0x974e bytes) 2024-04-24T18:09:03.187 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:03.187 INFO:teuthology.orchestra.run.smithi049.stdout:5729 clone 51 order 21 su 8192 sc 4 2024-04-24T18:09:03.267 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:03.267 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:03.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:03.267 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:03.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.096114876000001, 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.270 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.07647575, 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.272 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.274 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.276 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.076444027999997, 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:09:03.280 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.076420611000003, 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.281 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:09:03.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.076390537999998, 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:09:03.285 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:03.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075345390999999, 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:09:03.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.293 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075328159999998, 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:09:03.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:09:03.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:09:03.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:09:03.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075311161999998, 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.298 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.299 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075292480999998, 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.302 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:03.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:09:03.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:09:03.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075275013000002, 2024-04-24T18:09:03.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.310 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075258888999997, 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.314 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075227783000003, 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:09:03.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:09:03.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075208582000002, 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.075187540000002, 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:09:03.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.074432592000001, 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:09:03.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:09:03.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:09:03.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:09:03.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.074406089999997, 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:09:03.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.074388560000003, 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.335 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:03.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:09:03.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.072925400000003, 2024-04-24T18:09:03.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.343 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.061086752000001, 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:09:03.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:09:03.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.347 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.061070409000003, 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:03.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:03.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:09:03.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:03.351 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:03.352 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:03.354 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:03.355 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:03.362+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:03.455 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:03.463+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:03.479 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:03.479 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:03.479 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:03.480 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:03.489 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:03.556 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:03.563+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:03.612 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:03.612 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:03.612 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:03.612 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:03.612 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:03.622 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:03.657 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:03.664+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:03.747 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.415829493000004, 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.748 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:03.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:03.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.415813587000002, 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:03.752 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:03.756 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.415761619999998, 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.414838145999994, 2024-04-24T18:09:03.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:03.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:03.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.764 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.414821513999996, 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:03.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.767 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.768 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.414119911, 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.769 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:03.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.772 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.414103409000006, 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:03.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:03.776 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.413200543000002, 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:03.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.413192518000002, 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:03.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:03.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.785 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.412128375999998, 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.789 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.412112406999995, 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.790 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:03.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:03.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.793 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 37.351913967999998, 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:03.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:03.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:03.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:03.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.182889935999999, 2024-04-24T18:09:03.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.801 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.182873701999998, 2024-04-24T18:09:03.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:03.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.182858482, 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.807 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:03.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.809 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.182830902999999, 2024-04-24T18:09:03.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:03.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.815 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.182812601999998, 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.816 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:03.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.820 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.182796303, 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:03.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:03.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 9.8743936839999993, 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:03.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:03.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.5251205790000002, 2024-04-24T18:09:03.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:03.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:03.833 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:03.835 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:03.835 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:03.842+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:03.930 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:09:03.935 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:03.943+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:03.963 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:03.963 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:03.963 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:03.963 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:03.975 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:03.991 INFO:teuthology.orchestra.run.smithi049.stdout:leaving image image_client.0-clone50 intact 2024-04-24T18:09:04.036 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:04.044+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:04.098 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:04.099 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:04.099 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:04.099 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:04.099 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:04.109 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:04.143 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #49, image image_client.0-clone49 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent50 2024-04-24T18:09:04.152 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:04.161+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:04.238 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:04.246+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:04.239 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:04.239 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:04.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:04.239 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:04.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.239 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.239 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.882346843999997, 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:04.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:04.244 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.882338715000003, 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:04.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:04.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.882330691999996, 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:04.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:04.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.882321779000002, 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.254 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.261 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.88231296, 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:04.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:04.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:04.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.882303827999998, 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.266 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:04.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.270 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881483568999997, 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:04.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:04.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881456370000002, 2024-04-24T18:09:04.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:04.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.279 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881438899999999, 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.280 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:04.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:04.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881423560000002, 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.284 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:04.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881399481999999, 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.289 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:04.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.293 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881383917000001, 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:04.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881368956000003, 2024-04-24T18:09:04.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:04.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881352016999998, 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.303 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:04.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:04.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881335612000001, 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.308 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:04.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:04.312 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881319173000001, 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.313 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881304333000003, 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.318 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881287909000001, 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.322 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:04.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:04.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.326 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881270694000001, 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:04.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 33.881253229000002, 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:04.331 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:04.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.335 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:04.337 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:04.338 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:04.346+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:04.344 INFO:teuthology.orchestra.run.smithi049.stdout:5730 punch from 0x44ee13c to 0x44f8045, (0x9f09 bytes) 2024-04-24T18:09:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:5732 read 0x65d4b80 thru 0x65d4fd0 (0x451 bytes) 2024-04-24T18:09:04.350 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.363 INFO:teuthology.orchestra.run.smithi049.stdout:5735 trunc from 0xea9317a to 0x3100d3a 2024-04-24T18:09:04.439 INFO:teuthology.orchestra.run.smithi049.stdout:5736 read 0x1a5785e thru 0x1a62f8a (0xb72d bytes) 2024-04-24T18:09:04.439 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:04.447+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:04.445 INFO:teuthology.orchestra.run.smithi049.stdout:5737 trunc from 0x3100d3a to 0x8a3969b 2024-04-24T18:09:04.451 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.458 INFO:teuthology.orchestra.run.smithi049.stdout:5740 writesame 0x90d5e37 thru 0x90dc4a8 data_size 0x1f(0x6672 bytes) 2024-04-24T18:09:04.459 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.459 INFO:teuthology.orchestra.run.smithi049.stdout:5742 trunc from 0x90dc4a9 to 0x980fc02 2024-04-24T18:09:04.466 INFO:teuthology.orchestra.run.smithi049.stdout:5745 read 0x8ae56aa thru 0x8aeb653 (0x5faa bytes) 2024-04-24T18:09:04.467 INFO:teuthology.orchestra.run.smithi049.stdout:5746 punch from 0xd95f84 to 0xd9b646, (0x56c2 bytes) 2024-04-24T18:09:04.467 INFO:teuthology.orchestra.run.smithi049.stdout:5747 write 0x601e189 thru 0x602616e (0x7fe6 bytes) 2024-04-24T18:09:04.474 INFO:teuthology.orchestra.run.smithi049.stdout:5748 write 0x655cc10 thru 0x6568079 (0xb46a bytes) 2024-04-24T18:09:04.476 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.477 INFO:teuthology.orchestra.run.smithi049.stdout:5751 trunc from 0x980fc02 to 0x8c57fdd 2024-04-24T18:09:04.483 INFO:teuthology.orchestra.run.smithi049.stdout:5752 read 0xce03f1 thru 0xce4b02 (0x4712 bytes) 2024-04-24T18:09:04.484 INFO:teuthology.orchestra.run.smithi049.stdout:5753 write 0x53ecc6d thru 0x53f49c4 (0x7d58 bytes) 2024-04-24T18:09:04.504 INFO:teuthology.orchestra.run.smithi049.stdout:5755 write 0xeb30d4d thru 0xeb37627 (0x68db bytes) 2024-04-24T18:09:04.507 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:04.507 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:04.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:04.507 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:04.513 INFO:teuthology.orchestra.run.smithi049.stdout:5757 trunc from 0xeb37628 to 0xeb14661 2024-04-24T18:09:04.519 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:04.521 INFO:teuthology.orchestra.run.smithi049.stdout:5760 punch from 0x5cbc8e2 to 0x5cc7389, (0xaaa7 bytes) 2024-04-24T18:09:04.521 INFO:teuthology.orchestra.run.smithi049.stdout:5762 read 0xcc9970d thru 0xcc998db (0x1cf bytes) 2024-04-24T18:09:04.521 INFO:teuthology.orchestra.run.smithi049.stdout:5763 read 0x6ba2a3c thru 0x6baf530 (0xcaf5 bytes) 2024-04-24T18:09:04.521 INFO:teuthology.orchestra.run.smithi049.stdout:5764 write 0xaea611 thru 0xaf87ef (0xe1df bytes) 2024-04-24T18:09:04.547 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:04.552+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:04.640 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:04.649+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:04.641 INFO:teuthology.orchestra.run.smithi049.stdout:5765 write 0x95988e6 thru 0x95a821c (0xf937 bytes) 2024-04-24T18:09:04.646 INFO:teuthology.orchestra.run.smithi049.stdout:5766 read 0xab9058f thru 0xab9164a (0x10bc bytes) 2024-04-24T18:09:04.657 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:04.657 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:04.657 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:04.657 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:04.657 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:04.661 INFO:teuthology.orchestra.run.smithi049.stdout:5767 trunc from 0xeb14661 to 0x1db2525 2024-04-24T18:09:04.668 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:04.699 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.699 INFO:teuthology.orchestra.run.smithi049.stdout:5771 read 0x17fe051 thru 0x17ffcfb (0x1cab bytes) 2024-04-24T18:09:04.717 INFO:teuthology.orchestra.run.smithi049.stdout:5772 write 0xc455d63 thru 0xc465a20 (0xfcbe bytes) 2024-04-24T18:09:04.722 INFO:teuthology.orchestra.run.smithi049.stdout:5773 read 0x7be4035 thru 0x7bf3dce (0xfd9a bytes) 2024-04-24T18:09:04.724 INFO:teuthology.orchestra.run.smithi049.stdout:5774 trunc from 0xc465a21 to 0xd2a608c 2024-04-24T18:09:04.729 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.743 INFO:teuthology.orchestra.run.smithi049.stdout:5776 trunc from 0xd2a608c to 0xa03319 2024-04-24T18:09:04.751 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:04.760+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:04.757 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:04.757 INFO:teuthology.orchestra.run.smithi049.stdout:5779 read 0x2bd994 thru 0x2cc672 (0xecdf bytes) 2024-04-24T18:09:04.779 INFO:teuthology.orchestra.run.smithi049.stdout:5780 write 0xe2b4db2 thru 0xe2b5c0e (0xe5d bytes) 2024-04-24T18:09:04.783 INFO:teuthology.orchestra.run.smithi049.stdout:5781 read 0x7555ba5 thru 0x755b182 (0x55de bytes) 2024-04-24T18:09:04.783 INFO:teuthology.orchestra.run.smithi049.stdout:5783 read 0x29e0ccf thru 0x29ec9a5 (0xbcd7 bytes) 2024-04-24T18:09:04.783 INFO:teuthology.orchestra.run.smithi049.stdout:5784 read 0x3458a29 thru 0x346879d (0xfd75 bytes) 2024-04-24T18:09:04.783 INFO:teuthology.orchestra.run.smithi049.stdout:5785 write 0x8ddf985 thru 0x8de06d7 (0xd53 bytes) 2024-04-24T18:09:04.787 INFO:teuthology.orchestra.run.smithi049.stdout:5786 write 0xb7f55ac thru 0xb8004dc (0xaf31 bytes) 2024-04-24T18:09:04.795 INFO:teuthology.orchestra.run.smithi049.stdout:5787 writesame 0xd3c7c49 thru 0xd3cc24c data_size 0x2(0x4604 bytes) 2024-04-24T18:09:04.802 INFO:teuthology.orchestra.run.smithi049.stdout:5788 writesame 0x992fa21 thru 0x99325df data_size 0x1(0x2bbf bytes) 2024-04-24T18:09:04.822 INFO:teuthology.orchestra.run.smithi049.stdout:5789 trunc from 0xe2b5c0f to 0x4afc77a 2024-04-24T18:09:04.839 INFO:teuthology.orchestra.run.smithi049.stdout:5790 punch from 0x15d6c25 to 0x15db8a7, (0x4c82 bytes) 2024-04-24T18:09:04.842 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:04.850+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.234708744, 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.844 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.234684329, 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:04.849 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:04.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.854 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.23466118, 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:04.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.234580612, 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.860 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:04.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.864 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.234555802, 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.23453569, 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.870 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:04.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:04.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.234465027, 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.875 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 104.234453254, 2024-04-24T18:09:04.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:04.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:04.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.885 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 103.814131425, 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:04.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:04.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 103.814084243, 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:04.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:04.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 103.814059122, 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:04.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:04.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:04.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 103.814006988, 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.901 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:04.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:04.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 103.813943524, 2024-04-24T18:09:04.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:04.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:04.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 72.040444807, 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:04.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:04.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:04.915 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 72.039338737999998, 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:04.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:04.919 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 32.88416702, 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:04.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:04.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.8222130759999997, 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.923 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:04.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.8209259930000004, 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:04.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.930 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.4817499219999997, 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:04.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 4.481660228, 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:04.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:04.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:04.938 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:04.939 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:04.941 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5791 write 0x72d79cb thru 0x72d8ebd (0x14f3 bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5792 writesame 0xbf1765 thru 0xbfeaf1 data_size 0x1(0xd38d bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5793 read 0x2720a0d thru 0x2726b1d (0x6111 bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5794 writesame 0x3a07b58 thru 0x3a107c9 data_size 0x1(0x8c72 bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5796 writesame 0xc85ae3a thru 0xc86aa9d data_size 0x3a(0xfc64 bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5797 trunc from 0xc86aa9e to 0x80084b9 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5799 read 0x6f76dcb thru 0x6f77e35 (0x106b bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5800 writesame 0x2df25fc thru 0x2dfbda5 data_size 0x1b(0x97aa bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5801 trunc from 0x80084b9 to 0x2befd0d 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5803 read 0x4d8219 thru 0x4dbb71 (0x3959 bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5804 writesame 0x3e71cf9 thru 0x3e80942 data_size 0x73(0xec4a bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5805 punch from 0x35ef305 to 0x35fa505, (0xb200 bytes) 2024-04-24T18:09:04.942 INFO:teuthology.orchestra.run.smithi049.stdout:5806 punch from 0xfe168a to 0xfe415e, (0x2ad4 bytes) 2024-04-24T18:09:04.943 INFO:teuthology.orchestra.run.smithi049.stdout:5807 write 0x7a2f62 thru 0x7ac5ee (0x968d bytes) 2024-04-24T18:09:04.950 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:04.958+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:05.044 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:05.052+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:05.045 INFO:teuthology.orchestra.run.smithi049.stdout:5808 trunc from 0x3e80943 to 0x13e375c 2024-04-24T18:09:05.056 INFO:teuthology.orchestra.run.smithi049.stdout:5809 punch from 0x13da4bc to 0x13e375c, (0x92a0 bytes) 2024-04-24T18:09:05.074 INFO:teuthology.orchestra.run.smithi049.stdout:5810 writesame 0xbca07d4 thru 0xbcab532 data_size 0x1(0xad5f bytes) 2024-04-24T18:09:05.074 INFO:teuthology.orchestra.run.smithi049.stdout:5811 write 0xb32af50 thru 0xb33500f (0xa0c0 bytes) 2024-04-24T18:09:05.091 INFO:teuthology.orchestra.run.smithi049.stdout:5812 trunc from 0xbcab533 to 0x46b225b 2024-04-24T18:09:05.103 INFO:teuthology.orchestra.run.smithi049.stdout:5813 read 0x281411f thru 0x2823883 (0xf765 bytes) 2024-04-24T18:09:05.103 INFO:teuthology.orchestra.run.smithi049.stdout:5814 punch from 0xf99370 to 0xfa6c74, (0xd904 bytes) 2024-04-24T18:09:05.103 INFO:teuthology.orchestra.run.smithi049.stdout:5816 punch from 0x434abeb to 0x435a25b, (0xf670 bytes) 2024-04-24T18:09:05.103 INFO:teuthology.orchestra.run.smithi049.stdout:5817 read 0x17aba34 thru 0x17aef3c (0x3509 bytes) 2024-04-24T18:09:05.103 INFO:teuthology.orchestra.run.smithi049.stdout:5819 read 0x72a564 thru 0x734bf2 (0xa68f bytes) 2024-04-24T18:09:05.110 INFO:teuthology.orchestra.run.smithi049.stdout:5820 write 0x573ae33 thru 0x57482f2 (0xd4c0 bytes) 2024-04-24T18:09:05.115 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:05.115 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:05.115 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:05.115 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:05.116 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.116 INFO:teuthology.orchestra.run.smithi049.stdout:5822 read 0xf7c2c1 thru 0xf87015 (0xad55 bytes) 2024-04-24T18:09:05.116 INFO:teuthology.orchestra.run.smithi049.stdout:5823 punch from 0x328ad63 to 0x328dd6a, (0x3007 bytes) 2024-04-24T18:09:05.116 INFO:teuthology.orchestra.run.smithi049.stdout:5824 writesame 0x3f174c8 thru 0x3f1e7da data_size 0x1(0x7313 bytes) 2024-04-24T18:09:05.127 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:05.128 INFO:teuthology.orchestra.run.smithi049.stdout:5825 punch from 0x45e97c8 to 0x45f90cf, (0xf907 bytes) 2024-04-24T18:09:05.128 INFO:teuthology.orchestra.run.smithi049.stdout:5826 read 0x2d9b0e7 thru 0x2da236a (0x7284 bytes) 2024-04-24T18:09:05.128 INFO:teuthology.orchestra.run.smithi049.stdout:5827 write 0x1a32707 thru 0x1a3a191 (0x7a8b bytes) 2024-04-24T18:09:05.134 INFO:teuthology.orchestra.run.smithi049.stdout:5829 punch from 0x1daaf8c to 0x1dad30e, (0x2382 bytes) 2024-04-24T18:09:05.134 INFO:teuthology.orchestra.run.smithi049.stdout:5830 write 0x4a6e953 thru 0x4a71b29 (0x31d7 bytes) 2024-04-24T18:09:05.148 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:05.148 INFO:teuthology.orchestra.run.smithi049.stdout:5831 write 0xa5ece33 thru 0xa5efff9 (0x31c7 bytes) 2024-04-24T18:09:05.152 INFO:teuthology.orchestra.run.smithi049.stdout:5832 punch from 0x6c9872d to 0x6ca2d3f, (0xa612 bytes) 2024-04-24T18:09:05.153 INFO:teuthology.orchestra.run.smithi049.stdout:5833 read 0x2caedb2 thru 0x2cbb3a7 (0xc5f6 bytes) 2024-04-24T18:09:05.153 INFO:teuthology.orchestra.run.smithi049.stdout:5834 punch from 0x7e6d44c to 0x7e752d7, (0x7e8b bytes) 2024-04-24T18:09:05.155 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:09:05.159 INFO:teuthology.orchestra.run.smithi049.stdout:5835 trunc from 0xa5efffa to 0x44e8508 2024-04-24T18:09:05.172 INFO:teuthology.orchestra.run.smithi049.stdout:5836 punch from 0xcc0aca to 0xccd216, (0xc74c bytes) 2024-04-24T18:09:05.172 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:05.172 INFO:teuthology.orchestra.run.smithi049.stdout:5837 writesame 0x27fc856 thru 0x280030e data_size 0x1(0x3ab9 bytes) 2024-04-24T18:09:05.172 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:05.181+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:05.190 INFO:teuthology.orchestra.run.smithi049.stdout:5838 writesame 0x70c9f37 thru 0x70d8b1d data_size 0x1(0xebe7 bytes) 2024-04-24T18:09:05.190 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.190 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.190 INFO:teuthology.orchestra.run.smithi049.stdout:5842 read 0x101f22f thru 0x102bf85 (0xcd57 bytes) 2024-04-24T18:09:05.190 INFO:teuthology.orchestra.run.smithi049.stdout:5844 punch from 0x4b4075b to 0x4b41554, (0xdf9 bytes) 2024-04-24T18:09:05.190 INFO:teuthology.orchestra.run.smithi049.stdout:5845 read 0x345dbd2 thru 0x3460776 (0x2ba5 bytes) 2024-04-24T18:09:05.191 INFO:teuthology.orchestra.run.smithi049.stdout:5846 read 0x141b584 thru 0x1423434 (0x7eb1 bytes) 2024-04-24T18:09:05.191 INFO:teuthology.orchestra.run.smithi049.stdout:5848 punch from 0x4f84a30 to 0x4f89de6, (0x53b6 bytes) 2024-04-24T18:09:05.196 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:05.203 INFO:teuthology.orchestra.run.smithi049.stdout:5849 write 0xda800b6 thru 0xda874fd (0x7448 bytes) 2024-04-24T18:09:05.211 INFO:teuthology.orchestra.run.smithi049.stdout:5850 punch from 0xa82e5f3 to 0xa839ee2, (0xb8ef bytes) 2024-04-24T18:09:05.212 INFO:teuthology.orchestra.run.smithi049.stdout:5851 write 0x6ff7224 thru 0x700444b (0xd228 bytes) 2024-04-24T18:09:05.217 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:05.222 INFO:teuthology.orchestra.run.smithi049.stdout:5852 read 0x179b069 thru 0x17a81a5 (0xd13d bytes) 2024-04-24T18:09:05.227 INFO:teuthology.orchestra.run.smithi049.stdout:5854 write 0xe545b58 thru 0xe547e92 (0x233b bytes) 2024-04-24T18:09:05.231 INFO:teuthology.orchestra.run.smithi049.stdout:5855 write 0xb152326 thru 0xb157eea (0x5bc5 bytes) 2024-04-24T18:09:05.236 INFO:teuthology.orchestra.run.smithi049.stdout:5856 write 0x5cc0343 thru 0x5cc1f63 (0x1c21 bytes) 2024-04-24T18:09:05.239 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.239 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.239 INFO:teuthology.orchestra.run.smithi049.stdout:5862 read 0xd820b6 thru 0xd8f4c4 (0xd40f bytes) 2024-04-24T18:09:05.239 INFO:teuthology.orchestra.run.smithi049.stdout:5864 write 0x5f77059 thru 0x5f83825 (0xc7cd bytes) 2024-04-24T18:09:05.240 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:05.244 INFO:teuthology.orchestra.run.smithi049.stdout:5865 writesame 0x95d7170 thru 0x95de132 data_size 0x1(0x6fc3 bytes) 2024-04-24T18:09:05.246 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:05.254+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:05.258 INFO:teuthology.orchestra.run.smithi049.stdout:5866 punch from 0xaa52ca0 to 0xaa5ac1f, (0x7f7f bytes) 2024-04-24T18:09:05.258 INFO:teuthology.orchestra.run.smithi049.stdout:5867 write 0x90ad135 thru 0x90ba1fd (0xd0c9 bytes) 2024-04-24T18:09:05.264 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:05.267 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:05.267 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:05.267 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:05.267 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:05.267 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:05.268 INFO:teuthology.orchestra.run.smithi049.stdout:5868 trunc from 0xe547e93 to 0x9f87aaf 2024-04-24T18:09:05.279 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:05.284 INFO:teuthology.orchestra.run.smithi049.stdout:5869 read 0x7f9935a thru 0x7fa829e (0xef45 bytes) 2024-04-24T18:09:05.284 INFO:teuthology.orchestra.run.smithi049.stdout:5870 read 0x852a12d thru 0x85389f6 (0xe8ca bytes) 2024-04-24T18:09:05.284 INFO:teuthology.orchestra.run.smithi049.stdout:5871 punch from 0x8eb4503 to 0x8ec173c, (0xd239 bytes) 2024-04-24T18:09:05.287 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:05.293 INFO:teuthology.orchestra.run.smithi049.stdout:5872 trunc from 0x9f87aaf to 0x1765e07 2024-04-24T18:09:05.312 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:05.314 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:05.317 INFO:teuthology.orchestra.run.smithi049.stdout:5873 write 0x90a05f thru 0x915141 (0xb0e3 bytes) 2024-04-24T18:09:05.322 INFO:teuthology.orchestra.run.smithi049.stdout:5874 trunc from 0x1765e07 to 0x1c81265 2024-04-24T18:09:05.328 INFO:teuthology.orchestra.run.smithi049.stdout:5875 read 0x19b6ac6 thru 0x19b9815 (0x2d50 bytes) 2024-04-24T18:09:05.328 INFO:teuthology.orchestra.run.smithi049.stdout:5876 punch from 0x15237da to 0x1523bbb, (0x3e1 bytes) 2024-04-24T18:09:05.328 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.328 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.332 INFO:teuthology.orchestra.run.smithi049.stdout:5879 trunc from 0x1c81265 to 0x5a305ae 2024-04-24T18:09:05.347 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:05.355+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:05.353 INFO:teuthology.orchestra.run.smithi049.stdout:5881 write 0xdd60897 thru 0xdd65882 (0x4fec bytes) 2024-04-24T18:09:05.360 INFO:teuthology.orchestra.run.smithi049.stdout:5882 writesame 0x86cddca thru 0x86d6fb3 data_size 0x13(0x91ea bytes) 2024-04-24T18:09:05.377 INFO:teuthology.orchestra.run.smithi049.stdout:5883 trunc from 0xdd65883 to 0x52b3d91 2024-04-24T18:09:05.389 INFO:teuthology.orchestra.run.smithi049.stdout:5884 read 0x3c69972 thru 0x3c730b2 (0x9741 bytes) 2024-04-24T18:09:05.392 INFO:teuthology.orchestra.run.smithi049.stdout:5885 trunc from 0x52b3d91 to 0x7b3ff6e 2024-04-24T18:09:05.397 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.398 INFO:teuthology.orchestra.run.smithi049.stdout:5888 read 0x74bc414 thru 0x74c1852 (0x543f bytes) 2024-04-24T18:09:05.410 INFO:teuthology.orchestra.run.smithi049.stdout:5889 write 0xe2a7127 thru 0xe2b47e1 (0xd6bb bytes) 2024-04-24T18:09:05.418 INFO:teuthology.orchestra.run.smithi049.stdout:5891 write 0x60a1f52 thru 0x60b0bf7 (0xeca6 bytes) 2024-04-24T18:09:05.424 INFO:teuthology.orchestra.run.smithi049.stdout:5892 write 0x286eada thru 0x2870459 (0x1980 bytes) 2024-04-24T18:09:05.429 INFO:teuthology.orchestra.run.smithi049.stdout:5893 read 0x65dbde2 thru 0x65dfe47 (0x4066 bytes) 2024-04-24T18:09:05.436 INFO:teuthology.orchestra.run.smithi049.stdout:5897 write 0xea78783 thru 0xea85aa0 (0xd31e bytes) 2024-04-24T18:09:05.443 INFO:teuthology.orchestra.run.smithi049.stdout:5898 read 0x4649c65 thru 0x464c4da (0x2876 bytes) 2024-04-24T18:09:05.443 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.443 INFO:teuthology.orchestra.run.smithi049.stdout:5901 punch from 0x8ed8bbb to 0x8ee7442, (0xe887 bytes) 2024-04-24T18:09:05.443 INFO:teuthology.orchestra.run.smithi049.stdout:5903 read 0x4aabee8 thru 0x4ab8782 (0xc89b bytes) 2024-04-24T18:09:05.443 INFO:teuthology.orchestra.run.smithi049.stdout:5907 write 0xaf86307 thru 0xaf8aa68 (0x4762 bytes) 2024-04-24T18:09:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:5908 punch from 0xc38f0d6 to 0xc38ff10, (0xe3a bytes) 2024-04-24T18:09:05.448 INFO:teuthology.orchestra.run.smithi049.stdout:5910 write 0x82323c3 thru 0x82404f0 (0xe12e bytes) 2024-04-24T18:09:05.448 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:05.456+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:05.455 INFO:teuthology.orchestra.run.smithi049.stdout:5911 read 0xc94542c thru 0xc94b18e (0x5d63 bytes) 2024-04-24T18:09:05.455 INFO:teuthology.orchestra.run.smithi049.stdout:5912 write 0xbd67f31 thru 0xbd72c00 (0xacd0 bytes) 2024-04-24T18:09:05.458 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:05.458 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:05.461 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.461 INFO:teuthology.orchestra.run.smithi049.stdout:5915 write 0x9fae371 thru 0x9fb13d9 (0x3069 bytes) 2024-04-24T18:09:05.464 INFO:teuthology.orchestra.run.smithi049.stdout:5916 punch from 0xad28852 to 0xad2df95, (0x5743 bytes) 2024-04-24T18:09:05.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:05.475 INFO:teuthology.orchestra.run.smithi049.stdout:5918 trunc from 0xea85aa1 to 0x4f9cbb6 2024-04-24T18:09:05.494 INFO:teuthology.orchestra.run.smithi049.stdout:5919 write 0x36cafc0 thru 0x36da6a0 (0xf6e1 bytes) 2024-04-24T18:09:05.500 INFO:teuthology.orchestra.run.smithi049.stdout:5920 write 0x1526c8a thru 0x152f813 (0x8b8a bytes) 2024-04-24T18:09:05.506 INFO:teuthology.orchestra.run.smithi049.stdout:5921 write 0x58cf1a thru 0x58e260 (0x1347 bytes) 2024-04-24T18:09:05.507 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.511 INFO:teuthology.orchestra.run.smithi049.stdout:5923 trunc from 0x4f9cbb6 to 0x1f7f25c 2024-04-24T18:09:05.523 INFO:teuthology.orchestra.run.smithi049.stdout:5926 read 0x1791c01 thru 0x1795e7e (0x427e bytes) 2024-04-24T18:09:05.523 INFO:teuthology.orchestra.run.smithi049.stdout:5927 read 0x1d103a9 thru 0x1d1d1b5 (0xce0d bytes) 2024-04-24T18:09:05.523 INFO:teuthology.orchestra.run.smithi049.stdout:5928 read 0x13bc9f7 thru 0x13becd2 (0x22dc bytes) 2024-04-24T18:09:05.527 INFO:teuthology.orchestra.run.smithi049.stdout:5930 trunc from 0x1f7f25c to 0x523c535 2024-04-24T18:09:05.533 INFO:teuthology.orchestra.run.smithi049.stdout:5932 trunc from 0x523c535 to 0x4a275e2 2024-04-24T18:09:05.539 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.549 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:05.557+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:05.552 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:05.554 INFO:teuthology.orchestra.run.smithi049.stdout:5934 write 0xd00939a thru 0xd00d46d (0x40d4 bytes) 2024-04-24T18:09:05.554 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:05.560 INFO:teuthology.orchestra.run.smithi049.stdout:5935 write 0xa3ddd92 thru 0xa3e73c6 (0x9635 bytes) 2024-04-24T18:09:05.569 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-04-24T18:09:05.317609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1383014559,"num_objects":3802,"num_object_clones":696,"num_object_copies":7604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3800,"num_whiteouts":2353,"num_read":263184,"num_read_kb":786207,"num_write":25560,"num_write_kb":502482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21265,"ondisk_log_size":21265,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2506356,"kb_used_data":80784,"kb_used_omap":43,"kb_used_meta":2425044,"kb_avail":752468364,"statfs":{"total":773094113280,"available":770527604736,"internally_reserved":0,"allocated":82722816,"data_stored":687217310,"data_compressed":13443219,"data_compressed_allocated":44625920,"data_compressed_original":657905470,"omap_allocated":44665,"internal_metadata":2483245447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,9,0,5,41],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53836759,"num_objects":96,"num_object_clones":23,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":23,"num_read":36079,"num_read_kb":53066,"num_write":4553,"num_write_kb":28739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.400790"},"pg_stats":[{"pgid":"4.1f","version":"103'478","reported_seq":5787,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.846556+0000","last_change":"2024-04-24T18:08:56.910909+0000","last_active":"2024-04-24T18:09:01.846556+0000","last_peered":"2024-04-24T18:09:01.846556+0000","last_clean":"2024-04-24T18:09:01.846556+0000","last_became_active":"2024-04-24T18:08:56.900267+0000","last_became_peered":"2024-04-24T18:08:56.900267+0000","last_unstale":"2024-04-24T18:09:01.846556+0000","last_undegraded":"2024-04-24T18:09:01.846556+0000","last_fullsized":"2024-04-24T18:09:01.846556+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:07.953531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":48493296,"num_objects":140,"num_object_clones":25,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":94,"num_read":5800,"num_read_kb":28016,"num_write":480,"num_write_kb":11832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"103'1178","reported_seq":5768,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.268887+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:09:02.268887+0000","last_peered":"2024-04-24T18:09:02.268887+0000","last_clean":"2024-04-24T18:09:02.268887+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:09:02.268887+0000","last_undegraded":"2024-04-24T18:09:02.268887+0000","last_fullsized":"2024-04-24T18:09:02.268887+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1178,"log_dups_size":0,"ondisk_log_size":1178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":38892248,"num_objects":140,"num_object_clones":24,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":89,"num_read":9061,"num_read_kb":19501,"num_write":1749,"num_write_kb":9325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"103'555","reported_seq":5048,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343045+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:09:02.343045+0000","last_peered":"2024-04-24T18:09:02.343045+0000","last_clean":"2024-04-24T18:09:02.343045+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:09:02.343045+0000","last_undegraded":"2024-04-24T18:09:02.343045+0000","last_fullsized":"2024-04-24T18:09:02.343045+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":34147030,"num_objects":141,"num_object_clones":26,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":90,"num_read":5494,"num_read_kb":16748,"num_write":532,"num_write_kb":11979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"103'838","reported_seq":9063,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.284358+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:09:02.284358+0000","last_peered":"2024-04-24T18:09:02.284358+0000","last_clean":"2024-04-24T18:09:02.284358+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:09:02.284358+0000","last_undegraded":"2024-04-24T18:09:02.284358+0000","last_fullsized":"2024-04-24T18:09:02.284358+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":39935192,"num_objects":111,"num_object_clones":24,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":64,"num_read":11137,"num_read_kb":22432,"num_write":1164,"num_write_kb":9823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"103'442","reported_seq":11250,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.148826+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:09:02.148826+0000","last_peered":"2024-04-24T18:09:02.148826+0000","last_clean":"2024-04-24T18:09:02.148826+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:09:02.148826+0000","last_undegraded":"2024-04-24T18:09:02.148826+0000","last_fullsized":"2024-04-24T18:09:02.148826+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":52771434,"num_objects":120,"num_object_clones":28,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":65,"num_read":10909,"num_read_kb":37118,"num_write":449,"num_write_kb":19128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"103'477","reported_seq":10887,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.323918+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:09:02.323918+0000","last_peered":"2024-04-24T18:09:02.323918+0000","last_clean":"2024-04-24T18:09:02.323918+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:09:02.323918+0000","last_undegraded":"2024-04-24T18:09:02.323918+0000","last_fullsized":"2024-04-24T18:09:02.323918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":41098372,"num_objects":120,"num_object_clones":27,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":68,"num_read":11707,"num_read_kb":28307,"num_write":436,"num_write_kb":11686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"103'617","reported_seq":8127,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.313548+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:09:02.313548+0000","last_peered":"2024-04-24T18:09:02.313548+0000","last_clean":"2024-04-24T18:09:02.313548+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:09:02.313548+0000","last_undegraded":"2024-04-24T18:09:02.313548+0000","last_fullsized":"2024-04-24T18:09:02.313548+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":41397531,"num_objects":120,"num_object_clones":19,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":77,"num_read":8869,"num_read_kb":27108,"num_write":740,"num_write_kb":15831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"103'484","reported_seq":10525,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.340728+0000","last_change":"2024-04-24T18:08:57.909809+0000","last_active":"2024-04-24T18:09:02.340728+0000","last_peered":"2024-04-24T18:09:02.340728+0000","last_clean":"2024-04-24T18:09:02.340728+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:09:02.340728+0000","last_undegraded":"2024-04-24T18:09:02.340728+0000","last_fullsized":"2024-04-24T18:09:02.340728+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":46688097,"num_objects":108,"num_object_clones":19,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":61,"num_read":11293,"num_read_kb":28652,"num_write":504,"num_write_kb":28639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"103'1252","reported_seq":6172,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343546+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:09:02.343546+0000","last_peered":"2024-04-24T18:09:02.343546+0000","last_clean":"2024-04-24T18:09:02.343546+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:09:02.343546+0000","last_undegraded":"2024-04-24T18:09:02.343546+0000","last_fullsized":"2024-04-24T18:09:02.343546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":42133504,"num_objects":116,"num_object_clones":26,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":72,"num_read":9984,"num_read_kb":24510,"num_write":2014,"num_write_kb":17623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"103'1086","reported_seq":4335,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.236199+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:09:02.236199+0000","last_peered":"2024-04-24T18:09:02.236199+0000","last_clean":"2024-04-24T18:09:02.236199+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:09:02.236199+0000","last_undegraded":"2024-04-24T18:09:02.236199+0000","last_fullsized":"2024-04-24T18:09:02.236199+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1086,"log_dups_size":0,"ondisk_log_size":1086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":43652270,"num_objects":124,"num_object_clones":20,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":76,"num_read":8065,"num_read_kb":23692,"num_write":1599,"num_write_kb":17530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.923555+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:08:59.923555+0000","last_peered":"2024-04-24T18:08:59.923555+0000","last_clean":"2024-04-24T18:08:59.923555+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:08:59.923555+0000","last_undegraded":"2024-04-24T18:08:59.923555+0000","last_fullsized":"2024-04-24T18:08:59.923555+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.923789+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:08:59.923789+0000","last_peered":"2024-04-24T18:08:59.923789+0000","last_clean":"2024-04-24T18:08:59.923789+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:08:59.923789+0000","last_undegraded":"2024-04-24T18:08:59.923789+0000","last_fullsized":"2024-04-24T18:08:59.923789+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"103'607","reported_seq":5444,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.346495+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:09:02.346495+0000","last_peered":"2024-04-24T18:09:02.346495+0000","last_clean":"2024-04-24T18:09:02.346495+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:09:02.346495+0000","last_undegraded":"2024-04-24T18:09:02.346495+0000","last_fullsized":"2024-04-24T18:09:02.346495+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":59856909,"num_objects":124,"num_object_clones":24,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":74,"num_read":6345,"num_read_kb":31146,"num_write":642,"num_write_kb":18594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":92,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.918608+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:59.918608+0000","last_peered":"2024-04-24T18:08:59.918608+0000","last_clean":"2024-04-24T18:08:59.918608+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:59.918608+0000","last_undegraded":"2024-04-24T18:08:59.918608+0000","last_fullsized":"2024-04-24T18:08:59.918608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.923394+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:59.923394+0000","last_peered":"2024-04-24T18:08:59.923394+0000","last_clean":"2024-04-24T18:08:59.923394+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:59.923394+0000","last_undegraded":"2024-04-24T18:08:59.923394+0000","last_fullsized":"2024-04-24T18:08:59.923394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"103'890","reported_seq":5821,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.345427+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:09:02.345427+0000","last_peered":"2024-04-24T18:09:02.345427+0000","last_clean":"2024-04-24T18:09:02.345427+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:09:02.345427+0000","last_undegraded":"2024-04-24T18:09:02.345427+0000","last_fullsized":"2024-04-24T18:09:02.345427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":30848679,"num_objects":119,"num_object_clones":20,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":84,"num_read":8173,"num_read_kb":12939,"num_write":1157,"num_write_kb":16262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"61'1","reported_seq":93,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.916935+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:59.916935+0000","last_peered":"2024-04-24T18:08:59.916935+0000","last_clean":"2024-04-24T18:08:59.916935+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:59.916935+0000","last_undegraded":"2024-04-24T18:08:59.916935+0000","last_fullsized":"2024-04-24T18:08:59.916935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:06.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.919592+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:59.919592+0000","last_peered":"2024-04-24T18:08:59.919592+0000","last_clean":"2024-04-24T18:08:59.919592+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:59.919592+0000","last_undegraded":"2024-04-24T18:08:59.919592+0000","last_fullsized":"2024-04-24T18:08:59.919592+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"103'798","reported_seq":9416,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.869706+0000","last_change":"2024-04-24T18:08:56.912177+0000","last_active":"2024-04-24T18:09:01.869706+0000","last_peered":"2024-04-24T18:09:01.869706+0000","last_clean":"2024-04-24T18:09:01.869706+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:09:01.869706+0000","last_undegraded":"2024-04-24T18:09:01.869706+0000","last_fullsized":"2024-04-24T18:09:01.869706+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":60953345,"num_objects":149,"num_object_clones":24,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":91,"num_read":11354,"num_read_kb":43146,"num_write":964,"num_write_kb":32544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":66,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.909348+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:59.909348+0000","last_peered":"2024-04-24T18:08:59.909348+0000","last_clean":"2024-04-24T18:08:59.909348+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:59.909348+0000","last_undegraded":"2024-04-24T18:08:59.909348+0000","last_fullsized":"2024-04-24T18:08:59.909348+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.919501+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:59.919501+0000","last_peered":"2024-04-24T18:08:59.919501+0000","last_clean":"2024-04-24T18:08:59.919501+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:59.919501+0000","last_undegraded":"2024-04-24T18:08:59.919501+0000","last_fullsized":"2024-04-24T18:08:59.919501+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"103'425","reported_seq":4622,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.312554+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:09:02.312554+0000","last_peered":"2024-04-24T18:09:02.312554+0000","last_clean":"2024-04-24T18:09:02.312554+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:09:02.312554+0000","last_undegraded":"2024-04-24T18:09:02.312554+0000","last_fullsized":"2024-04-24T18:09:02.312554+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":31171510,"num_objects":115,"num_object_clones":28,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":66,"num_read":5810,"num_read_kb":24092,"num_write":554,"num_write_kb":12828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'1","reported_seq":93,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.911789+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:59.911789+0000","last_peered":"2024-04-24T18:08:59.911789+0000","last_clean":"2024-04-24T18:08:59.911789+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:59.911789+0000","last_undegraded":"2024-04-24T18:08:59.911789+0000","last_fullsized":"2024-04-24T18:08:59.911789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":292,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.919595+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:59.919595+0000","last_peered":"2024-04-24T18:08:59.919595+0000","last_clean":"2024-04-24T18:08:59.919595+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:59.919595+0000","last_undegraded":"2024-04-24T18:08:59.919595+0000","last_fullsized":"2024-04-24T18:08:59.919595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":81,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.909445+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:08:59.909445+0000","last_peered":"2024-04-24T18:08:59.909445+0000","last_clean":"2024-04-24T18:08:59.909445+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:08:59.909445+0000","last_undegraded":"2024-04-24T18:08:59.909445+0000","last_fullsized":"2024-04-24T18:08:59.909445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"103'813","reported_seq":8785,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.267575+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:09:02.267575+0000","last_peered":"2024-04-24T18:09:02.267575+0000","last_clean":"2024-04-24T18:09:02.267575+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:09:02.267575+0000","last_undegraded":"2024-04-24T18:09:02.267575+0000","last_fullsized":"2024-04-24T18:09:02.267575+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":43562429,"num_objects":99,"num_object_clones":16,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":58,"num_read":11606,"num_read_kb":30422,"num_write":1054,"num_write_kb":22026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"103'538","reported_seq":5529,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.285915+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:09:02.285915+0000","last_peered":"2024-04-24T18:09:02.285915+0000","last_clean":"2024-04-24T18:09:02.285915+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:09:02.285915+0000","last_undegraded":"2024-04-24T18:09:02.285915+0000","last_fullsized":"2024-04-24T18:09:02.285915+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":48776479,"num_objects":116,"num_object_clones":18,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":83,"num_read":6306,"num_read_kb":29404,"num_write":460,"num_write_kb":23472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.917162+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:59.917162+0000","last_peered":"2024-04-24T18:08:59.917162+0000","last_clean":"2024-04-24T18:08:59.917162+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:59.917162+0000","last_undegraded":"2024-04-24T18:08:59.917162+0000","last_fullsized":"2024-04-24T18:08:59.917162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"103'667","reported_seq":5165,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343596+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:09:02.343596+0000","last_peered":"2024-04-24T18:09:02.343596+0000","last_clean":"2024-04-24T18:09:02.343596+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:09:02.343596+0000","last_undegraded":"2024-04-24T18:09:02.343596+0000","last_fullsized":"2024-04-24T18:09:02.343596+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":41365544,"num_objects":110,"num_object_clones":19,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":62,"num_read":6632,"num_read_kb":15966,"num_write":735,"num_write_kb":7535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.918746+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:59.918746+0000","last_peered":"2024-04-24T18:08:59.918746+0000","last_clean":"2024-04-24T18:08:59.918746+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:59.918746+0000","last_undegraded":"2024-04-24T18:08:59.918746+0000","last_fullsized":"2024-04-24T18:08:59.918746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"103'530","reported_seq":10625,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343296+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:09:02.343296+0000","last_peered":"2024-04-24T18:09:02.343296+0000","last_clean":"2024-04-24T18:09:02.343296+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:09:02.343296+0000","last_undegraded":"2024-04-24T18:09:02.343296+0000","last_fullsized":"2024-04-24T18:09:02.343296+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":46055361,"num_objects":130,"num_object_clones":25,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":79,"num_read":11258,"num_read_kb":36195,"num_write":549,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"103'566","reported_seq":5873,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.286680+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:09:02.286680+0000","last_peered":"2024-04-24T18:09:02.286680+0000","last_clean":"2024-04-24T18:09:02.286680+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:09:02.286680+0000","last_undegraded":"2024-04-24T18:09:02.286680+0000","last_fullsized":"2024-04-24T18:09:02.286680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":46300149,"num_objects":117,"num_object_clones":19,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":72,"num_read":6034,"num_read_kb":18004,"num_write":613,"num_write_kb":12781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"103'585","reported_seq":3504,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.754170+0000","last_change":"2024-04-24T18:08:56.908403+0000","last_active":"2024-04-24T18:09:01.754170+0000","last_peered":"2024-04-24T18:09:01.754170+0000","last_clean":"2024-04-24T18:09:01.754170+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:09:01.754170+0000","last_undegraded":"2024-04-24T18:09:01.754170+0000","last_fullsized":"2024-04-24T18:09:01.754170+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":26766610,"num_objects":86,"num_object_clones":8,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":60,"num_read":3715,"num_read_kb":11520,"num_write":669,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"103'535","reported_seq":4495,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.267592+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:09:02.267592+0000","last_peered":"2024-04-24T18:09:02.267592+0000","last_clean":"2024-04-24T18:09:02.267592+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:09:02.267592+0000","last_undegraded":"2024-04-24T18:09:02.267592+0000","last_fullsized":"2024-04-24T18:09:02.267592+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":25458313,"num_objects":97,"num_object_clones":15,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":63,"num_read":6679,"num_read_kb":13978,"num_write":583,"num_write_kb":11146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"103'483","reported_seq":6263,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.345254+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:09:02.345254+0000","last_peered":"2024-04-24T18:09:02.345254+0000","last_clean":"2024-04-24T18:09:02.345254+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:09:02.345254+0000","last_undegraded":"2024-04-24T18:09:02.345254+0000","last_fullsized":"2024-04-24T18:09:02.345254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":40085453,"num_objects":96,"num_object_clones":15,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":61,"num_read":6997,"num_read_kb":27430,"num_write":519,"num_write_kb":28588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"103'500","reported_seq":8449,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.007365+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:09:02.007365+0000","last_peered":"2024-04-24T18:09:02.007365+0000","last_clean":"2024-04-24T18:09:02.007365+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:09:02.007365+0000","last_undegraded":"2024-04-24T18:09:02.007365+0000","last_fullsized":"2024-04-24T18:09:02.007365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":47647052,"num_objects":135,"num_object_clones":25,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":85,"num_read":8436,"num_read_kb":21093,"num_write":472,"num_write_kb":7417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"103'479","reported_seq":6391,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.316126+0000","last_change":"2024-04-24T18:08:56.920936+0000","last_active":"2024-04-24T18:09:02.316126+0000","last_peered":"2024-04-24T18:09:02.316126+0000","last_clean":"2024-04-24T18:09:02.316126+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:09:02.316126+0000","last_undegraded":"2024-04-24T18:09:02.316126+0000","last_fullsized":"2024-04-24T18:09:02.316126+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":43117226,"num_objects":112,"num_object_clones":24,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":63,"num_read":6320,"num_read_kb":15328,"num_write":510,"num_write_kb":13493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"103'700","reported_seq":5069,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.276625+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:09:02.276625+0000","last_peered":"2024-04-24T18:09:02.276625+0000","last_clean":"2024-04-24T18:09:02.276625+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:09:02.276625+0000","last_undegraded":"2024-04-24T18:09:02.276625+0000","last_fullsized":"2024-04-24T18:09:02.276625+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":37383530,"num_objects":122,"num_object_clones":18,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":73,"num_read":7063,"num_read_kb":18236,"num_write":808,"num_write_kb":8501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"103'526","reported_seq":5932,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.345301+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:09:02.345301+0000","last_peered":"2024-04-24T18:09:02.345301+0000","last_clean":"2024-04-24T18:09:02.345301+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:09:02.345301+0000","last_undegraded":"2024-04-24T18:09:02.345301+0000","last_fullsized":"2024-04-24T18:09:02.345301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":29538394,"num_objects":128,"num_object_clones":24,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":79,"num_read":6709,"num_read_kb":15669,"num_write":582,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"103'658","reported_seq":8337,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.268649+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:09:02.268649+0000","last_peered":"2024-04-24T18:09:02.268649+0000","last_clean":"2024-04-24T18:09:02.268649+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:09:02.268649+0000","last_undegraded":"2024-04-24T18:09:02.268649+0000","last_fullsized":"2024-04-24T18:09:02.268649+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":95480404,"num_objects":132,"num_object_clones":27,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":84,"num_read":8987,"num_read_kb":47185,"num_write":815,"num_write_kb":30856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"103'543","reported_seq":5478,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.317463+0000","last_change":"2024-04-24T18:08:56.897040+0000","last_active":"2024-04-24T18:09:02.317463+0000","last_peered":"2024-04-24T18:09:02.317463+0000","last_clean":"2024-04-24T18:09:02.317463+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:09:02.317463+0000","last_undegraded":"2024-04-24T18:09:02.317463+0000","last_fullsized":"2024-04-24T18:09:02.317463+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":26740303,"num_objects":122,"num_object_clones":16,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":82,"num_read":9157,"num_read_kb":29349,"num_write":514,"num_write_kb":8693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"103'878","reported_seq":8727,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.236422+0000","last_change":"2024-04-24T18:08:57.908349+0000","last_active":"2024-04-24T18:09:02.236422+0000","last_peered":"2024-04-24T18:09:02.236422+0000","last_clean":"2024-04-24T18:09:02.236422+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:09:02.236422+0000","last_undegraded":"2024-04-24T18:09:02.236422+0000","last_fullsized":"2024-04-24T18:09:02.236422+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":43298546,"num_objects":110,"num_object_clones":26,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":67,"num_read":10314,"num_read_kb":22641,"num_write":1251,"num_write_kb":22554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"103'511","reported_seq":7025,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.316505+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:09:02.316505+0000","last_peered":"2024-04-24T18:09:02.316505+0000","last_clean":"2024-04-24T18:09:02.316505+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:09:02.316505+0000","last_undegraded":"2024-04-24T18:09:02.316505+0000","last_fullsized":"2024-04-24T18:09:02.316505+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":49500631,"num_objects":97,"num_object_clones":20,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":57,"num_read":8788,"num_read_kb":22905,"num_write":456,"num_write_kb":7659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"103'488","reported_seq":5213,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.994189+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:09:01.994189+0000","last_peered":"2024-04-24T18:09:01.994189+0000","last_clean":"2024-04-24T18:09:01.994189+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:09:01.994189+0000","last_undegraded":"2024-04-24T18:09:01.994189+0000","last_fullsized":"2024-04-24T18:09:01.994189+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":20515746,"num_objects":108,"num_object_clones":20,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":69,"num_read":6753,"num_read_kb":12219,"num_write":421,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"103'1019","reported_seq":5388,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.276962+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:09:02.276962+0000","last_peered":"2024-04-24T18:09:02.276962+0000","last_clean":"2024-04-24T18:09:02.276962+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:09:02.276962+0000","last_undegraded":"2024-04-24T18:09:02.276962+0000","last_fullsized":"2024-04-24T18:09:02.276962+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":58792604,"num_objects":134,"num_object_clones":27,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":83,"num_read":7313,"num_read_kb":31156,"num_write":1406,"num_write_kb":16836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1382424191,"num_objects":3798,"num_object_clones":696,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":2351,"num_read":263068,"num_read_kb":786107,"num_write":25401,"num_write_kb":500066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72110080,"data_stored":678712447,"data_compressed":13414601,"data_compressed_allocated":44142592,"data_compressed_original":654545500,"omap_allocated":9738,"internal_metadata":0},"log_size":21146,"ondisk_log_size":21146,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305580,"kb_used_data":8228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066260,"statfs":{"total":96636764160,"available":96323850240,"internally_reserved":0,"allocated":8425472,"data_stored":70117677,"data_compressed":1044543,"data_compressed_allocated":4341760,"data_compressed_original":67180196,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.308}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42999999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.495},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.36899999999999999}]}]},{"osd":6,"up_from":95,"seq":408021893123,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311428,"kb_used_data":8336,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94060412,"statfs":{"total":96636764160,"available":96317861888,"internally_reserved":0,"allocated":8536064,"data_stored":70139488,"data_compressed":1641079,"data_compressed_allocated":5005312,"data_compressed_original":67485365,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,9],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.071999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3940000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2669999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323996,"kb_used_data":14344,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94047844,"statfs":{"total":96636764160,"available":96304992256,"internally_reserved":0,"allocated":14688256,"data_stored":103287088,"data_compressed":2046500,"data_compressed_allocated":7024640,"data_compressed_original":97280867,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.15,"5min":14.15,"15min":14.15},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":14.089,"5min":14.089,"15min":14.089},"last":0.443},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.622}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.74299999999999999}]},{"osd":7,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":14.051,"5min":14.051,"15min":14.051},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304308,"kb_used_data":6940,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067532,"statfs":{"total":96636764160,"available":96325152768,"internally_reserved":0,"allocated":7106560,"data_stored":63194073,"data_compressed":1118263,"data_compressed_allocated":3981312,"data_compressed_original":61014924,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":26.936,"5min":26.936,"15min":26.936},"last":0.27700000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":14.371,"5min":14.371,"15min":14.371},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.973000000000001,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.414,"5min":14.414,"15min":14.414},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":14.452,"5min":14.452,"15min":14.452},"last":0.74299999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":14.486000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314084,"kb_used_data":4432,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94057756,"statfs":{"total":96636764160,"available":96315142144,"internally_reserved":0,"allocated":4538368,"data_stored":27972243,"data_compressed":569852,"data_compressed_allocated":2011136,"data_compressed_original":26273097,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.32600000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.377},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.505},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.433}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325448,"kb_used_data":15796,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94046392,"statfs":{"total":96636764160,"available":96303505408,"internally_reserved":0,"allocated":16175104,"data_stored":144912107,"data_compressed":2418292,"data_compressed_allocated":8945664,"data_compressed_original":139038187,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,41],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":2.8119999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":2.8999999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":4.6699999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":3.2360000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":1.3129999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":3.2130000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":3.4049999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":2.7389999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":2.7669999999999999}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":2.919},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":15.4}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314036,"kb_used_data":12588,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94057804,"statfs":{"total":96636764160,"available":96315191296,"internally_reserved":0,"allocated":12890112,"data_stored":123235305,"data_compressed":2758162,"data_compressed_allocated":7639040,"data_compressed_original":118906964,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.434},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.314}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.74199999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.35099999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.47499999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.48699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307476,"kb_used_data":10120,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064364,"statfs":{"total":96636764160,"available":96321908736,"internally_reserved":0,"allocated":10362880,"data_stored":84359329,"data_compressed":1846528,"data_compressed_allocated":5677056,"data_compressed_original":80725870,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.501},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42099999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.33600000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.78200000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.623}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.81200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5775360,"data_stored":62204812,"data_compressed":1116682,"data_compressed_allocated":3919872,"data_compressed_original":60670860,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13225984,"data_stored":101530905,"data_compressed":2034639,"data_compressed_allocated":6901760,"data_compressed_original":96231666,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3223552,"data_stored":26999265,"data_compressed":568185,"data_compressed_allocated":1941504,"data_compressed_original":25924937,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14749696,"data_stored":143661236,"data_compressed":2415587,"data_compressed_allocated":8835072,"data_compressed_original":138468218,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11685888,"data_stored":122641948,"data_compressed":2758162,"data_compressed_allocated":7639040,"data_compressed_original":118906964,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9101312,"data_stored":83616540,"data_compressed":1845985,"data_compressed_allocated":5656576,"data_compressed_original":80602990,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7364608,"data_stored":69572826,"data_compressed":1641079,"data_compressed_allocated":5005312,"data_compressed_original":67481269,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6983680,"data_stored":68484915,"data_compressed":1034282,"data_compressed_allocated":4243456,"data_compressed_original":66258596,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:05.572 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-24T18:09:05.572 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:05.573 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:09:05.573 INFO:teuthology.orchestra.run.smithi049.stdout:5936 write 0xdb54c17 thru 0xdb57d03 (0x30ed bytes) 2024-04-24T18:09:05.579 INFO:teuthology.orchestra.run.smithi049.stdout:5937 read 0x47dcf9 thru 0x486f0c (0x9214 bytes) 2024-04-24T18:09:05.580 INFO:teuthology.orchestra.run.smithi049.stdout:5938 read 0xd67e03e thru 0xd684d65 (0x6d28 bytes) 2024-04-24T18:09:05.581 INFO:teuthology.orchestra.run.smithi049.stdout:5942 writesame 0xa7be6d7 thru 0xa7cceea data_size 0x6(0xe814 bytes) 2024-04-24T18:09:05.587 INFO:teuthology.orchestra.run.smithi049.stdout:5943 write 0x83f8701 thru 0x83fa3e1 (0x1ce1 bytes) 2024-04-24T18:09:05.591 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:05.591 INFO:teuthology.orchestra.run.smithi049.stdout:5945 punch from 0x54c09c1 to 0x54ca0ca, (0x9709 bytes) 2024-04-24T18:09:05.591 INFO:teuthology.orchestra.run.smithi049.stdout:5946 read 0x2e2df2d thru 0x2e3118d (0x3261 bytes) 2024-04-24T18:09:05.591 INFO:teuthology.orchestra.run.smithi049.stdout:5947 write 0x23450d5 thru 0x23528ad (0xd7d9 bytes) 2024-04-24T18:09:05.597 INFO:teuthology.orchestra.run.smithi049.stdout:5948 read 0x26bf62a thru 0x26ce860 (0xf237 bytes) 2024-04-24T18:09:05.610 INFO:teuthology.orchestra.run.smithi049.stdout:5949 trunc from 0xdb57d04 to 0x26e15e1 2024-04-24T18:09:05.628 INFO:teuthology.orchestra.run.smithi049.stdout:5950 punch from 0x105499a to 0x106189a, (0xcf00 bytes) 2024-04-24T18:09:05.635 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:05.635 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:05.641 INFO:teuthology.orchestra.run.smithi049.stdout:5951 write 0x97984ad thru 0x979a5f5 (0x2149 bytes) 2024-04-24T18:09:05.645 INFO:teuthology.orchestra.run.smithi049.stdout:5952 punch from 0x274bb9 to 0x277650, (0x2a97 bytes) 2024-04-24T18:09:05.648 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:05.650 INFO:teuthology.orchestra.run.smithi049.stdout:5954 trunc from 0x979a5f6 to 0xd879385 2024-04-24T18:09:05.651 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:05.658+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:05.655 INFO:teuthology.orchestra.run.smithi049.stdout:5956 write 0xc370cd6 thru 0xc37fdeb (0xf116 bytes) 2024-04-24T18:09:05.661 INFO:teuthology.orchestra.run.smithi049.stdout:5957 punch from 0x489b832 to 0x48a258b, (0x6d59 bytes) 2024-04-24T18:09:05.661 INFO:teuthology.orchestra.run.smithi049.stdout:5958 write 0x8dcbf15 thru 0x8dd8ef9 (0xcfe5 bytes) 2024-04-24T18:09:05.667 INFO:teuthology.orchestra.run.smithi049.stdout:5959 clone 52 order 25 su 4194304 sc 6 2024-04-24T18:09:05.752 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:05.760+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:05.785 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:05.785 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:05.795 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:05.879 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:05.887+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:05.931 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:05.932 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:05.946 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":152,"stamp":"2024-04-24T18:09:05.317609+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1383014559,"num_objects":3802,"num_object_clones":696,"num_object_copies":7604,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3800,"num_whiteouts":2353,"num_read":263184,"num_read_kb":786207,"num_write":25560,"num_write_kb":502482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":21265,"ondisk_log_size":21265,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2506356,"kb_used_data":80784,"kb_used_omap":43,"kb_used_meta":2425044,"kb_avail":752468364,"statfs":{"total":773094113280,"available":770527604736,"internally_reserved":0,"allocated":82722816,"data_stored":687217310,"data_compressed":13443219,"data_compressed_allocated":44625920,"data_compressed_original":657905470,"omap_allocated":44665,"internal_metadata":2483245447},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,9,0,5,41],"upper_bound":32},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":53836759,"num_objects":96,"num_object_clones":23,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":23,"num_read":36079,"num_read_kb":53066,"num_write":4553,"num_write_kb":28739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.400790"},"pg_stats":[{"pgid":"4.1f","version":"103'478","reported_seq":5787,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.846556+0000","last_change":"2024-04-24T18:08:56.910909+0000","last_active":"2024-04-24T18:09:01.846556+0000","last_peered":"2024-04-24T18:09:01.846556+0000","last_clean":"2024-04-24T18:09:01.846556+0000","last_became_active":"2024-04-24T18:08:56.900267+0000","last_became_peered":"2024-04-24T18:08:56.900267+0000","last_unstale":"2024-04-24T18:09:01.846556+0000","last_undegraded":"2024-04-24T18:09:01.846556+0000","last_fullsized":"2024-04-24T18:09:01.846556+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":478,"log_dups_size":0,"ondisk_log_size":478,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:07.953531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":48493296,"num_objects":140,"num_object_clones":25,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":94,"num_read":5800,"num_read_kb":28016,"num_write":480,"num_write_kb":11832,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"103'1178","reported_seq":5768,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.268887+0000","last_change":"2024-04-24T18:07:38.289234+0000","last_active":"2024-04-24T18:09:02.268887+0000","last_peered":"2024-04-24T18:09:02.268887+0000","last_clean":"2024-04-24T18:09:02.268887+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:09:02.268887+0000","last_undegraded":"2024-04-24T18:09:02.268887+0000","last_fullsized":"2024-04-24T18:09:02.268887+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1178,"log_dups_size":0,"ondisk_log_size":1178,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044951499999999999,"stat_sum":{"num_bytes":38892248,"num_objects":140,"num_object_clones":24,"num_object_copies":280,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":140,"num_whiteouts":89,"num_read":9061,"num_read_kb":19501,"num_write":1749,"num_write_kb":9325,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"103'555","reported_seq":5048,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343045+0000","last_change":"2024-04-24T18:07:56.832284+0000","last_active":"2024-04-24T18:09:02.343045+0000","last_peered":"2024-04-24T18:09:02.343045+0000","last_clean":"2024-04-24T18:09:02.343045+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:09:02.343045+0000","last_undegraded":"2024-04-24T18:09:02.343045+0000","last_fullsized":"2024-04-24T18:09:02.343045+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":555,"log_dups_size":0,"ondisk_log_size":555,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00059792799999999996,"stat_sum":{"num_bytes":34147030,"num_objects":141,"num_object_clones":26,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":90,"num_read":5494,"num_read_kb":16748,"num_write":532,"num_write_kb":11979,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"103'838","reported_seq":9063,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.284358+0000","last_change":"2024-04-24T18:07:56.801935+0000","last_active":"2024-04-24T18:09:02.284358+0000","last_peered":"2024-04-24T18:09:02.284358+0000","last_clean":"2024-04-24T18:09:02.284358+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:09:02.284358+0000","last_undegraded":"2024-04-24T18:09:02.284358+0000","last_fullsized":"2024-04-24T18:09:02.284358+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":838,"log_dups_size":0,"ondisk_log_size":838,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00086446899999999998,"stat_sum":{"num_bytes":39935192,"num_objects":111,"num_object_clones":24,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":64,"num_read":11137,"num_read_kb":22432,"num_write":1164,"num_write_kb":9823,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"103'442","reported_seq":11250,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.148826+0000","last_change":"2024-04-24T18:07:38.291822+0000","last_active":"2024-04-24T18:09:02.148826+0000","last_peered":"2024-04-24T18:09:02.148826+0000","last_clean":"2024-04-24T18:09:02.148826+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:09:02.148826+0000","last_undegraded":"2024-04-24T18:09:02.148826+0000","last_fullsized":"2024-04-24T18:09:02.148826+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":442,"log_dups_size":0,"ondisk_log_size":442,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00072192399999999998,"stat_sum":{"num_bytes":52771434,"num_objects":120,"num_object_clones":28,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":65,"num_read":10909,"num_read_kb":37118,"num_write":449,"num_write_kb":19128,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"103'477","reported_seq":10887,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.323918+0000","last_change":"2024-04-24T18:07:38.292070+0000","last_active":"2024-04-24T18:09:02.323918+0000","last_peered":"2024-04-24T18:09:02.323918+0000","last_clean":"2024-04-24T18:09:02.323918+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:09:02.323918+0000","last_undegraded":"2024-04-24T18:09:02.323918+0000","last_fullsized":"2024-04-24T18:09:02.323918+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":477,"log_dups_size":0,"ondisk_log_size":477,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001056347,"stat_sum":{"num_bytes":41098372,"num_objects":120,"num_object_clones":27,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":68,"num_read":11707,"num_read_kb":28307,"num_write":436,"num_write_kb":11686,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"103'617","reported_seq":8127,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.313548+0000","last_change":"2024-04-24T18:07:38.293831+0000","last_active":"2024-04-24T18:09:02.313548+0000","last_peered":"2024-04-24T18:09:02.313548+0000","last_clean":"2024-04-24T18:09:02.313548+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:09:02.313548+0000","last_undegraded":"2024-04-24T18:09:02.313548+0000","last_fullsized":"2024-04-24T18:09:02.313548+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":617,"log_dups_size":0,"ondisk_log_size":617,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056274999999999999,"stat_sum":{"num_bytes":41397531,"num_objects":120,"num_object_clones":19,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":77,"num_read":8869,"num_read_kb":27108,"num_write":740,"num_write_kb":15831,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"103'484","reported_seq":10525,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.340728+0000","last_change":"2024-04-24T18:08:57.909809+0000","last_active":"2024-04-24T18:09:02.340728+0000","last_peered":"2024-04-24T18:09:02.340728+0000","last_clean":"2024-04-24T18:09:02.340728+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:09:02.340728+0000","last_undegraded":"2024-04-24T18:09:02.340728+0000","last_fullsized":"2024-04-24T18:09:02.340728+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":484,"log_dups_size":0,"ondisk_log_size":484,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000363741,"stat_sum":{"num_bytes":46688097,"num_objects":108,"num_object_clones":19,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":61,"num_read":11293,"num_read_kb":28652,"num_write":504,"num_write_kb":28639,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"103'1252","reported_seq":6172,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343546+0000","last_change":"2024-04-24T18:07:38.294313+0000","last_active":"2024-04-24T18:09:02.343546+0000","last_peered":"2024-04-24T18:09:02.343546+0000","last_clean":"2024-04-24T18:09:02.343546+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:09:02.343546+0000","last_undegraded":"2024-04-24T18:09:02.343546+0000","last_fullsized":"2024-04-24T18:09:02.343546+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1252,"log_dups_size":0,"ondisk_log_size":1252,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071021500000000004,"stat_sum":{"num_bytes":42133504,"num_objects":116,"num_object_clones":26,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":72,"num_read":9984,"num_read_kb":24510,"num_write":2014,"num_write_kb":17623,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"103'1086","reported_seq":4335,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.236199+0000","last_change":"2024-04-24T18:07:56.708257+0000","last_active":"2024-04-24T18:09:02.236199+0000","last_peered":"2024-04-24T18:09:02.236199+0000","last_clean":"2024-04-24T18:09:02.236199+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:09:02.236199+0000","last_undegraded":"2024-04-24T18:09:02.236199+0000","last_fullsized":"2024-04-24T18:09:02.236199+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1086,"log_dups_size":0,"ondisk_log_size":1086,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00043590000000000002,"stat_sum":{"num_bytes":43652270,"num_objects":124,"num_object_clones":20,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":76,"num_read":8065,"num_read_kb":23692,"num_write":1599,"num_write_kb":17530,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":69,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.923555+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:08:59.923555+0000","last_peered":"2024-04-24T18:08:59.923555+0000","last_clean":"2024-04-24T18:08:59.923555+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:08:59.923555+0000","last_undegraded":"2024-04-24T18:08:59.923555+0000","last_fullsized":"2024-04-24T18:08:59.923555+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":69,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.923789+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:08:59.923789+0000","last_peered":"2024-04-24T18:08:59.923789+0000","last_clean":"2024-04-24T18:08:59.923789+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:08:59.923789+0000","last_undegraded":"2024-04-24T18:08:59.923789+0000","last_fullsized":"2024-04-24T18:08:59.923789+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"103'607","reported_seq":5444,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.346495+0000","last_change":"2024-04-24T18:07:38.289023+0000","last_active":"2024-04-24T18:09:02.346495+0000","last_peered":"2024-04-24T18:09:02.346495+0000","last_clean":"2024-04-24T18:09:02.346495+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:09:02.346495+0000","last_undegraded":"2024-04-24T18:09:02.346495+0000","last_fullsized":"2024-04-24T18:09:02.346495+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":607,"log_dups_size":0,"ondisk_log_size":607,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000280022,"stat_sum":{"num_bytes":59856909,"num_objects":124,"num_object_clones":24,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":74,"num_read":6345,"num_read_kb":31146,"num_write":642,"num_write_kb":18594,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":92,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.918608+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:08:59.918608+0000","last_peered":"2024-04-24T18:08:59.918608+0000","last_clean":"2024-04-24T18:08:59.918608+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:08:59.918608+0000","last_undegraded":"2024-04-24T18:08:59.918608+0000","last_fullsized":"2024-04-24T18:08:59.918608+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.923394+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:08:59.923394+0000","last_peered":"2024-04-24T18:08:59.923394+0000","last_clean":"2024-04-24T18:08:59.923394+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:08:59.923394+0000","last_undegraded":"2024-04-24T18:08:59.923394+0000","last_fullsized":"2024-04-24T18:08:59.923394+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"103'890","reported_seq":5821,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.345427+0000","last_change":"2024-04-24T18:07:56.829946+0000","last_active":"2024-04-24T18:09:02.345427+0000","last_peered":"2024-04-24T18:09:02.345427+0000","last_clean":"2024-04-24T18:09:02.345427+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:09:02.345427+0000","last_undegraded":"2024-04-24T18:09:02.345427+0000","last_fullsized":"2024-04-24T18:09:02.345427+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":890,"log_dups_size":0,"ondisk_log_size":890,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00034244100000000002,"stat_sum":{"num_bytes":30848679,"num_objects":119,"num_object_clones":20,"num_object_copies":238,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":119,"num_whiteouts":84,"num_read":8173,"num_read_kb":12939,"num_write":1157,"num_write_kb":16262,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"61'1","reported_seq":93,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.916935+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:08:59.916935+0000","last_peered":"2024-04-24T18:08:59.916935+0000","last_clean":"2024-04-24T18:08:59.916935+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:08:59.916935+0000","last_undegraded":"2024-04-24T18:08:59.916935+0000","last_fullsized":"2024-04-24T18:08:59.916935+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:06.697341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":94,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.919592+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:08:59.919592+0000","last_peered":"2024-04-24T18:08:59.919592+0000","last_clean":"2024-04-24T18:08:59.919592+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:08:59.919592+0000","last_undegraded":"2024-04-24T18:08:59.919592+0000","last_fullsized":"2024-04-24T18:08:59.919592+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"103'798","reported_seq":9416,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.869706+0000","last_change":"2024-04-24T18:08:56.912177+0000","last_active":"2024-04-24T18:09:01.869706+0000","last_peered":"2024-04-24T18:09:01.869706+0000","last_clean":"2024-04-24T18:09:01.869706+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:09:01.869706+0000","last_undegraded":"2024-04-24T18:09:01.869706+0000","last_fullsized":"2024-04-24T18:09:01.869706+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":798,"log_dups_size":0,"ondisk_log_size":798,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":60953345,"num_objects":149,"num_object_clones":24,"num_object_copies":298,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":149,"num_whiteouts":91,"num_read":11354,"num_read_kb":43146,"num_write":964,"num_write_kb":32544,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":66,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.909348+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:08:59.909348+0000","last_peered":"2024-04-24T18:08:59.909348+0000","last_clean":"2024-04-24T18:08:59.909348+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:08:59.909348+0000","last_undegraded":"2024-04-24T18:08:59.909348+0000","last_fullsized":"2024-04-24T18:08:59.909348+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":94,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.919501+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:08:59.919501+0000","last_peered":"2024-04-24T18:08:59.919501+0000","last_clean":"2024-04-24T18:08:59.919501+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:08:59.919501+0000","last_undegraded":"2024-04-24T18:08:59.919501+0000","last_fullsized":"2024-04-24T18:08:59.919501+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"103'425","reported_seq":4622,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.312554+0000","last_change":"2024-04-24T18:07:56.798878+0000","last_active":"2024-04-24T18:09:02.312554+0000","last_peered":"2024-04-24T18:09:02.312554+0000","last_clean":"2024-04-24T18:09:02.312554+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:09:02.312554+0000","last_undegraded":"2024-04-24T18:09:02.312554+0000","last_fullsized":"2024-04-24T18:09:02.312554+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":425,"log_dups_size":0,"ondisk_log_size":425,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000251549,"stat_sum":{"num_bytes":31171510,"num_objects":115,"num_object_clones":28,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":66,"num_read":5810,"num_read_kb":24092,"num_write":554,"num_write_kb":12828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"61'1","reported_seq":93,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.911789+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:08:59.911789+0000","last_peered":"2024-04-24T18:08:59.911789+0000","last_clean":"2024-04-24T18:08:59.911789+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:08:59.911789+0000","last_undegraded":"2024-04-24T18:08:59.911789+0000","last_fullsized":"2024-04-24T18:08:59.911789+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":292,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.919595+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:08:59.919595+0000","last_peered":"2024-04-24T18:08:59.919595+0000","last_clean":"2024-04-24T18:08:59.919595+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:08:59.919595+0000","last_undegraded":"2024-04-24T18:08:59.919595+0000","last_fullsized":"2024-04-24T18:08:59.919595+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":81,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.909445+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:08:59.909445+0000","last_peered":"2024-04-24T18:08:59.909445+0000","last_clean":"2024-04-24T18:08:59.909445+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:08:59.909445+0000","last_undegraded":"2024-04-24T18:08:59.909445+0000","last_fullsized":"2024-04-24T18:08:59.909445+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"103'813","reported_seq":8785,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.267575+0000","last_change":"2024-04-24T18:07:59.651526+0000","last_active":"2024-04-24T18:09:02.267575+0000","last_peered":"2024-04-24T18:09:02.267575+0000","last_clean":"2024-04-24T18:09:02.267575+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:09:02.267575+0000","last_undegraded":"2024-04-24T18:09:02.267575+0000","last_fullsized":"2024-04-24T18:09:02.267575+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":813,"log_dups_size":0,"ondisk_log_size":813,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047255099999999998,"stat_sum":{"num_bytes":43562429,"num_objects":99,"num_object_clones":16,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":58,"num_read":11606,"num_read_kb":30422,"num_write":1054,"num_write_kb":22026,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"103'538","reported_seq":5529,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.285915+0000","last_change":"2024-04-24T18:07:38.291541+0000","last_active":"2024-04-24T18:09:02.285915+0000","last_peered":"2024-04-24T18:09:02.285915+0000","last_clean":"2024-04-24T18:09:02.285915+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:09:02.285915+0000","last_undegraded":"2024-04-24T18:09:02.285915+0000","last_fullsized":"2024-04-24T18:09:02.285915+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":538,"log_dups_size":0,"ondisk_log_size":538,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052303199999999999,"stat_sum":{"num_bytes":48776479,"num_objects":116,"num_object_clones":18,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":83,"num_read":6306,"num_read_kb":29404,"num_write":460,"num_write_kb":23472,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.917162+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:08:59.917162+0000","last_peered":"2024-04-24T18:08:59.917162+0000","last_clean":"2024-04-24T18:08:59.917162+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:08:59.917162+0000","last_undegraded":"2024-04-24T18:08:59.917162+0000","last_fullsized":"2024-04-24T18:08:59.917162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"103'667","reported_seq":5165,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343596+0000","last_change":"2024-04-24T18:07:38.293543+0000","last_active":"2024-04-24T18:09:02.343596+0000","last_peered":"2024-04-24T18:09:02.343596+0000","last_clean":"2024-04-24T18:09:02.343596+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:09:02.343596+0000","last_undegraded":"2024-04-24T18:09:02.343596+0000","last_fullsized":"2024-04-24T18:09:02.343596+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":667,"log_dups_size":0,"ondisk_log_size":667,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00029971799999999998,"stat_sum":{"num_bytes":41365544,"num_objects":110,"num_object_clones":19,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":62,"num_read":6632,"num_read_kb":15966,"num_write":735,"num_write_kb":7535,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":95,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-04-24T18:08:59.918746+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:08:59.918746+0000","last_peered":"2024-04-24T18:08:59.918746+0000","last_clean":"2024-04-24T18:08:59.918746+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:08:59.918746+0000","last_undegraded":"2024-04-24T18:08:59.918746+0000","last_fullsized":"2024-04-24T18:08:59.918746+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"103'530","reported_seq":10625,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.343296+0000","last_change":"2024-04-24T18:07:38.299655+0000","last_active":"2024-04-24T18:09:02.343296+0000","last_peered":"2024-04-24T18:09:02.343296+0000","last_clean":"2024-04-24T18:09:02.343296+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:09:02.343296+0000","last_undegraded":"2024-04-24T18:09:02.343296+0000","last_fullsized":"2024-04-24T18:09:02.343296+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":530,"log_dups_size":0,"ondisk_log_size":530,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000470008,"stat_sum":{"num_bytes":46055361,"num_objects":130,"num_object_clones":25,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":79,"num_read":11258,"num_read_kb":36195,"num_write":549,"num_write_kb":16636,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"103'566","reported_seq":5873,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.286680+0000","last_change":"2024-04-24T18:07:56.769050+0000","last_active":"2024-04-24T18:09:02.286680+0000","last_peered":"2024-04-24T18:09:02.286680+0000","last_clean":"2024-04-24T18:09:02.286680+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:09:02.286680+0000","last_undegraded":"2024-04-24T18:09:02.286680+0000","last_fullsized":"2024-04-24T18:09:02.286680+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00032672799999999998,"stat_sum":{"num_bytes":46300149,"num_objects":117,"num_object_clones":19,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":72,"num_read":6034,"num_read_kb":18004,"num_write":613,"num_write_kb":12781,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"103'585","reported_seq":3504,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.754170+0000","last_change":"2024-04-24T18:08:56.908403+0000","last_active":"2024-04-24T18:09:01.754170+0000","last_peered":"2024-04-24T18:09:01.754170+0000","last_clean":"2024-04-24T18:09:01.754170+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:09:01.754170+0000","last_undegraded":"2024-04-24T18:09:01.754170+0000","last_fullsized":"2024-04-24T18:09:01.754170+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":585,"log_dups_size":0,"ondisk_log_size":585,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":26766610,"num_objects":86,"num_object_clones":8,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":60,"num_read":3715,"num_read_kb":11520,"num_write":669,"num_write_kb":11333,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"103'535","reported_seq":4495,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.267592+0000","last_change":"2024-04-24T18:07:56.658843+0000","last_active":"2024-04-24T18:09:02.267592+0000","last_peered":"2024-04-24T18:09:02.267592+0000","last_clean":"2024-04-24T18:09:02.267592+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:09:02.267592+0000","last_undegraded":"2024-04-24T18:09:02.267592+0000","last_fullsized":"2024-04-24T18:09:02.267592+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037968600000000001,"stat_sum":{"num_bytes":25458313,"num_objects":97,"num_object_clones":15,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":63,"num_read":6679,"num_read_kb":13978,"num_write":583,"num_write_kb":11146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"103'483","reported_seq":6263,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.345254+0000","last_change":"2024-04-24T18:07:38.292250+0000","last_active":"2024-04-24T18:09:02.345254+0000","last_peered":"2024-04-24T18:09:02.345254+0000","last_clean":"2024-04-24T18:09:02.345254+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:09:02.345254+0000","last_undegraded":"2024-04-24T18:09:02.345254+0000","last_fullsized":"2024-04-24T18:09:02.345254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":483,"log_dups_size":0,"ondisk_log_size":483,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00098416799999999989,"stat_sum":{"num_bytes":40085453,"num_objects":96,"num_object_clones":15,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":61,"num_read":6997,"num_read_kb":27430,"num_write":519,"num_write_kb":28588,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"103'500","reported_seq":8449,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.007365+0000","last_change":"2024-04-24T18:07:38.292067+0000","last_active":"2024-04-24T18:09:02.007365+0000","last_peered":"2024-04-24T18:09:02.007365+0000","last_clean":"2024-04-24T18:09:02.007365+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:09:02.007365+0000","last_undegraded":"2024-04-24T18:09:02.007365+0000","last_fullsized":"2024-04-24T18:09:02.007365+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":500,"log_dups_size":0,"ondisk_log_size":500,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00088232999999999996,"stat_sum":{"num_bytes":47647052,"num_objects":135,"num_object_clones":25,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":85,"num_read":8436,"num_read_kb":21093,"num_write":472,"num_write_kb":7417,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"103'479","reported_seq":6391,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.316126+0000","last_change":"2024-04-24T18:08:56.920936+0000","last_active":"2024-04-24T18:09:02.316126+0000","last_peered":"2024-04-24T18:09:02.316126+0000","last_clean":"2024-04-24T18:09:02.316126+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:09:02.316126+0000","last_undegraded":"2024-04-24T18:09:02.316126+0000","last_fullsized":"2024-04-24T18:09:02.316126+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":479,"log_dups_size":0,"ondisk_log_size":479,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047093100000000001,"stat_sum":{"num_bytes":43117226,"num_objects":112,"num_object_clones":24,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":63,"num_read":6320,"num_read_kb":15328,"num_write":510,"num_write_kb":13493,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"103'700","reported_seq":5069,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.276625+0000","last_change":"2024-04-24T18:07:56.735773+0000","last_active":"2024-04-24T18:09:02.276625+0000","last_peered":"2024-04-24T18:09:02.276625+0000","last_clean":"2024-04-24T18:09:02.276625+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:09:02.276625+0000","last_undegraded":"2024-04-24T18:09:02.276625+0000","last_fullsized":"2024-04-24T18:09:02.276625+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":700,"log_dups_size":0,"ondisk_log_size":700,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052374500000000005,"stat_sum":{"num_bytes":37383530,"num_objects":122,"num_object_clones":18,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":73,"num_read":7063,"num_read_kb":18236,"num_write":808,"num_write_kb":8501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"103'526","reported_seq":5932,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.345301+0000","last_change":"2024-04-24T18:07:56.737928+0000","last_active":"2024-04-24T18:09:02.345301+0000","last_peered":"2024-04-24T18:09:02.345301+0000","last_clean":"2024-04-24T18:09:02.345301+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:09:02.345301+0000","last_undegraded":"2024-04-24T18:09:02.345301+0000","last_fullsized":"2024-04-24T18:09:02.345301+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046338400000000001,"stat_sum":{"num_bytes":29538394,"num_objects":128,"num_object_clones":24,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":79,"num_read":6709,"num_read_kb":15669,"num_write":582,"num_write_kb":9863,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"103'658","reported_seq":8337,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.268649+0000","last_change":"2024-04-24T18:07:38.293628+0000","last_active":"2024-04-24T18:09:02.268649+0000","last_peered":"2024-04-24T18:09:02.268649+0000","last_clean":"2024-04-24T18:09:02.268649+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:09:02.268649+0000","last_undegraded":"2024-04-24T18:09:02.268649+0000","last_fullsized":"2024-04-24T18:09:02.268649+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00037830900000000001,"stat_sum":{"num_bytes":95480404,"num_objects":132,"num_object_clones":27,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":84,"num_read":8987,"num_read_kb":47185,"num_write":815,"num_write_kb":30856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"103'543","reported_seq":5478,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.317463+0000","last_change":"2024-04-24T18:08:56.897040+0000","last_active":"2024-04-24T18:09:02.317463+0000","last_peered":"2024-04-24T18:09:02.317463+0000","last_clean":"2024-04-24T18:09:02.317463+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:09:02.317463+0000","last_undegraded":"2024-04-24T18:09:02.317463+0000","last_fullsized":"2024-04-24T18:09:02.317463+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046819199999999998,"stat_sum":{"num_bytes":26740303,"num_objects":122,"num_object_clones":16,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":82,"num_read":9157,"num_read_kb":29349,"num_write":514,"num_write_kb":8693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"103'878","reported_seq":8727,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.236422+0000","last_change":"2024-04-24T18:08:57.908349+0000","last_active":"2024-04-24T18:09:02.236422+0000","last_peered":"2024-04-24T18:09:02.236422+0000","last_clean":"2024-04-24T18:09:02.236422+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:09:02.236422+0000","last_undegraded":"2024-04-24T18:09:02.236422+0000","last_fullsized":"2024-04-24T18:09:02.236422+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":878,"log_dups_size":0,"ondisk_log_size":878,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096810699999999995,"stat_sum":{"num_bytes":43298546,"num_objects":110,"num_object_clones":26,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":67,"num_read":10314,"num_read_kb":22641,"num_write":1251,"num_write_kb":22554,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"103'511","reported_seq":7025,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.316505+0000","last_change":"2024-04-24T18:07:56.851503+0000","last_active":"2024-04-24T18:09:02.316505+0000","last_peered":"2024-04-24T18:09:02.316505+0000","last_clean":"2024-04-24T18:09:02.316505+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:09:02.316505+0000","last_undegraded":"2024-04-24T18:09:02.316505+0000","last_fullsized":"2024-04-24T18:09:02.316505+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":511,"log_dups_size":0,"ondisk_log_size":511,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050373900000000001,"stat_sum":{"num_bytes":49500631,"num_objects":97,"num_object_clones":20,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":57,"num_read":8788,"num_read_kb":22905,"num_write":456,"num_write_kb":7659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"103'488","reported_seq":5213,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:01.994189+0000","last_change":"2024-04-24T18:07:38.299984+0000","last_active":"2024-04-24T18:09:01.994189+0000","last_peered":"2024-04-24T18:09:01.994189+0000","last_clean":"2024-04-24T18:09:01.994189+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:09:01.994189+0000","last_undegraded":"2024-04-24T18:09:01.994189+0000","last_fullsized":"2024-04-24T18:09:01.994189+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":488,"log_dups_size":0,"ondisk_log_size":488,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000481848,"stat_sum":{"num_bytes":20515746,"num_objects":108,"num_object_clones":20,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":69,"num_read":6753,"num_read_kb":12219,"num_write":421,"num_write_kb":7053,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"103'1019","reported_seq":5388,"reported_epoch":103,"state":"active+clean","last_fresh":"2024-04-24T18:09:02.276962+0000","last_change":"2024-04-24T18:07:38.289217+0000","last_active":"2024-04-24T18:09:02.276962+0000","last_peered":"2024-04-24T18:09:02.276962+0000","last_clean":"2024-04-24T18:09:02.276962+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:09:02.276962+0000","last_undegraded":"2024-04-24T18:09:02.276962+0000","last_fullsized":"2024-04-24T18:09:02.276962+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1019,"log_dups_size":0,"ondisk_log_size":1019,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00041991499999999998,"stat_sum":{"num_bytes":58792604,"num_objects":134,"num_object_clones":27,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":83,"num_read":7313,"num_read_kb":31156,"num_write":1406,"num_write_kb":16836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1382424191,"num_objects":3798,"num_object_clones":696,"num_object_copies":7596,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3798,"num_whiteouts":2351,"num_read":263068,"num_read_kb":786107,"num_write":25401,"num_write_kb":500066,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":52,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72110080,"data_stored":678712447,"data_compressed":13414601,"data_compressed_allocated":44142592,"data_compressed_original":654545500,"omap_allocated":9738,"internal_metadata":0},"log_size":21146,"ondisk_log_size":21146,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574872,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305580,"kb_used_data":8228,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94066260,"statfs":{"total":96636764160,"available":96323850240,"internally_reserved":0,"allocated":8425472,"data_stored":70117677,"data_compressed":1044543,"data_compressed_allocated":4341760,"data_compressed_original":67180196,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.23999999999999999},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.34399999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.308}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.46899999999999997},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42999999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.44}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.45100000000000001}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.495},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.36899999999999999}]}]},{"osd":6,"up_from":95,"seq":408021893123,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311428,"kb_used_data":8336,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94060412,"statfs":{"total":96636764160,"available":96317861888,"internally_reserved":0,"allocated":8536064,"data_stored":70139488,"data_compressed":1641079,"data_compressed_allocated":5005312,"data_compressed_original":67485365,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[8,9],"upper_bound":4},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54100000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":10.071999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.3940000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.468}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.94199999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.2669999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574871,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":323996,"kb_used_data":14344,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94047844,"statfs":{"total":96636764160,"available":96304992256,"internally_reserved":0,"allocated":14688256,"data_stored":103287088,"data_compressed":2046500,"data_compressed_allocated":7024640,"data_compressed_original":97280867,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.30499999999999999},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.15,"5min":14.15,"15min":14.15},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.66700000000000004},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":0.60399999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":14.089,"5min":14.089,"15min":14.089},"last":0.443},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.46899999999999997}]},{"osd":4,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.622}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.74299999999999999}]},{"osd":7,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":14.051,"5min":14.051,"15min":14.051},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.53800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574872,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304308,"kb_used_data":6940,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067532,"statfs":{"total":96636764160,"available":96325152768,"internally_reserved":0,"allocated":7106560,"data_stored":63194073,"data_compressed":1118263,"data_compressed_allocated":3981312,"data_compressed_original":61014924,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.48199999999999998},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":26.936,"5min":26.936,"15min":26.936},"last":0.27700000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":1.0629999999999999},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.66000000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":14.371,"5min":14.371,"15min":14.371},"last":0.84699999999999998},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.973000000000001,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.57999999999999996}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.414,"5min":14.414,"15min":14.414},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.69499999999999995},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":14.452,"5min":14.452,"15min":14.452},"last":0.74299999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.55100000000000005}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"last":0.90800000000000003},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":14.486000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.81200000000000006}]}]},{"osd":2,"up_from":13,"seq":55834574871,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314084,"kb_used_data":4432,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94057756,"statfs":{"total":96636764160,"available":96315142144,"internally_reserved":0,"allocated":4538368,"data_stored":27972243,"data_compressed":569852,"data_compressed_allocated":2011136,"data_compressed_original":26273097,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.21199999999999999},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.32600000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.377},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.47799999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.505},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.433}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.55800000000000005}]}]},{"osd":3,"up_from":13,"seq":55834574872,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325448,"kb_used_data":15796,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94046392,"statfs":{"total":96636764160,"available":96303505408,"internally_reserved":0,"allocated":16175104,"data_stored":144912107,"data_compressed":2418292,"data_compressed_allocated":8945664,"data_compressed_original":139038187,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[0,0,0,5,41],"upper_bound":32},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":2.8119999999999998},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.34200000000000003}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":1.2829999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":2.8999999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":1.2490000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":4.6699999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":3.2360000000000002},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":1.3129999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":3.2130000000000001},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":3.4049999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":2.7389999999999999},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":2.7669999999999999}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":2.919},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":15.4}]}]},{"osd":4,"up_from":13,"seq":55834574872,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314036,"kb_used_data":12588,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94057804,"statfs":{"total":96636764160,"available":96315191296,"internally_reserved":0,"allocated":12890112,"data_stored":123235305,"data_compressed":2758162,"data_compressed_allocated":7639040,"data_compressed_original":118906964,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.377}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.434},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.314}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.53800000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.74199999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.35099999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.47499999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.48699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574872,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307476,"kb_used_data":10120,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064364,"statfs":{"total":96636764160,"available":96321908736,"internally_reserved":0,"allocated":10362880,"data_stored":84359329,"data_compressed":1846528,"data_compressed_allocated":5677056,"data_compressed_original":80725870,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.501},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.42099999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.33600000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.78200000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.74299999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.623}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.81200000000000006}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":5775360,"data_stored":62204812,"data_compressed":1116682,"data_compressed_allocated":3919872,"data_compressed_original":60670860,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13225984,"data_stored":101530905,"data_compressed":2034639,"data_compressed_allocated":6901760,"data_compressed_original":96231666,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3223552,"data_stored":26999265,"data_compressed":568185,"data_compressed_allocated":1941504,"data_compressed_original":25924937,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14749696,"data_stored":143661236,"data_compressed":2415587,"data_compressed_allocated":8835072,"data_compressed_original":138468218,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11685888,"data_stored":122641948,"data_compressed":2758162,"data_compressed_allocated":7639040,"data_compressed_original":118906964,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9101312,"data_stored":83616540,"data_compressed":1845985,"data_compressed_allocated":5656576,"data_compressed_original":80602990,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7364608,"data_stored":69572826,"data_compressed":1641079,"data_compressed_allocated":5005312,"data_compressed_original":67481269,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":6983680,"data_stored":68484915,"data_compressed":1034282,"data_compressed_allocated":4243456,"data_compressed_original":66258596,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:05.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:09:05.971 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:05.971 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:05.972 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:05.979+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:05.984 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:05.995 INFO:teuthology.orchestra.run.smithi049.stdout:truncating image image_client.0-clone51 from 0xd879385 (overlap 0xa03319) to 0x902dfc 2024-04-24T18:09:06.054 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:06.063+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:06.085 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #50, image image_client.0-clone50 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent51 2024-04-24T18:09:06.156 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:06.164+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:06.166 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:06.166 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:06.174 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:06.256 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:06.264+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:06.358 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:06.365+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:06.396 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:06.396 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:06.407 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:06.459 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:06.467+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:06.522 INFO:teuthology.orchestra.run.smithi049.stdout:5960 write 0xccf840d thru 0xcd05eaf (0xdaa3 bytes) 2024-04-24T18:09:06.529 INFO:teuthology.orchestra.run.smithi049.stdout:5961 read 0x212b546 thru 0x212bb72 (0x62d bytes) 2024-04-24T18:09:06.530 INFO:teuthology.orchestra.run.smithi049.stdout:5962 writesame 0x3d6498a thru 0x3d680d4 data_size 0x13(0x374b bytes) 2024-04-24T18:09:06.534 INFO:teuthology.orchestra.run.smithi049.stdout:5963 punch from 0xd35e8dd to 0xd3642ff, (0x5a22 bytes) 2024-04-24T18:09:06.534 INFO:teuthology.orchestra.run.smithi049.stdout:5964 write 0x81c1d7 thru 0x824e23 (0x8c4d bytes) 2024-04-24T18:09:06.553 INFO:teuthology.orchestra.run.smithi049.stdout:5965 trunc from 0xd879385 to 0x749a5f0 2024-04-24T18:09:06.560 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:06.568+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:06.583 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:06.583 INFO:teuthology.orchestra.run.smithi049.stdout:5967 punch from 0x56fba0b to 0x56fd9bc, (0x1fb1 bytes) 2024-04-24T18:09:06.584 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:06.584 INFO:teuthology.orchestra.run.smithi049.stdout:5969 write 0x3d29dfa thru 0x3d34f4d (0xb154 bytes) 2024-04-24T18:09:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:5972 punch from 0x708d216 to 0x708dae7, (0x8d1 bytes) 2024-04-24T18:09:06.585 INFO:teuthology.orchestra.run.smithi049.stdout:5973 write 0x6db849 thru 0x6dc4f7 (0xcaf bytes) 2024-04-24T18:09:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:06.586 INFO:teuthology.orchestra.run.smithi049.stdout:5975 read 0xd29cb4 thru 0xd39767 (0xfab4 bytes) 2024-04-24T18:09:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:5976 read 0x3b770d9 thru 0x3b78706 (0x162e bytes) 2024-04-24T18:09:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:5978 punch from 0x254d978 to 0x2555313, (0x799b bytes) 2024-04-24T18:09:06.587 INFO:teuthology.orchestra.run.smithi049.stdout:5984 read 0x68cfb6c thru 0x68d61d7 (0x666c bytes) 2024-04-24T18:09:06.592 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:06.592 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:06.594 INFO:teuthology.orchestra.run.smithi049.stdout:5986 trunc from 0x749a5f0 to 0x1c77bf6 2024-04-24T18:09:06.606 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:06.621 INFO:teuthology.orchestra.run.smithi049.stdout:5987 write 0xe1ede86 thru 0xe1f088a (0x2a05 bytes) 2024-04-24T18:09:06.625 INFO:teuthology.orchestra.run.smithi049.stdout:5988 write 0x35a1ddb thru 0x35a8edb (0x7101 bytes) 2024-04-24T18:09:06.626 INFO:teuthology.orchestra.run.smithi049.stdout:5989 read 0x7909b42 thru 0x79161bf (0xc67e bytes) 2024-04-24T18:09:06.627 INFO:teuthology.orchestra.run.smithi049.stdout:5990 write 0xa42dc98 thru 0xa42ea00 (0xd69 bytes) 2024-04-24T18:09:06.628 INFO:teuthology.orchestra.run.smithi049.stdout:5991 writesame 0xe0bfa5b thru 0xe0c2983 data_size 0x1(0x2f29 bytes) 2024-04-24T18:09:06.635 INFO:teuthology.orchestra.run.smithi049.stdout:5992 read 0x3718ad4 thru 0x371f3e2 (0x690f bytes) 2024-04-24T18:09:06.648 INFO:teuthology.orchestra.run.smithi049.stdout:5993 trunc from 0xe1f088b to 0x355bb2a 2024-04-24T18:09:06.661 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:06.669+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:06.677 INFO:teuthology.orchestra.run.smithi049.stdout:5994 writesame 0xc4e7808 thru 0xc4f1326 data_size 0x15(0x9b1f bytes) 2024-04-24T18:09:06.677 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:06.677 INFO:teuthology.orchestra.run.smithi049.stdout:5998 writesame 0xaab6c73 thru 0xaab7615 data_size 0x1(0x9a3 bytes) 2024-04-24T18:09:06.679 INFO:teuthology.orchestra.run.smithi049.stdout:compare and write disabled 2024-04-24T18:09:06.679 INFO:teuthology.orchestra.run.smithi049.stdout:6000 writesame 0x7cb3b22 thru 0x7cc3a37 data_size 0x2(0xff16 bytes) 2024-04-24T18:09:06.724 INFO:teuthology.orchestra.run.smithi049.stdout:checking clone #51, image image_client.0-clone51 against file /home/ubuntu/cephtest/archive/fsx-image_client.0-parent52 2024-04-24T18:09:06.754 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:06.755 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:06.762 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:06.770+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:06.766 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:06.863 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:06.871+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:06.941 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:09:06.953 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:06.954 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:06.964 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:06.965 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:06.972+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:07.065 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:07.073+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:07.139 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:07.139 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:07.149 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:07.166 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:07.174+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:07.266 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:07.275+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:07.278 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:07.278 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:07.288 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:07.328 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:07.375 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:07.383+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:07.419 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:07.419 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:07.430 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:07.471 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:07.479+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:07.477 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:07.496 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:07.515 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:07.534 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:07.554 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:07.569 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:07.577+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:07.573 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:07.592 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:07.603 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:07.603 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:07.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.431651262000003, 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.604 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:09:07.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.607 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.412012136000001, 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.608 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:09:07.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.611 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.411980413999999, 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.612 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:09:07.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.411956996999997, 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.411926923999999, 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:09:07.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410881777, 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410864545999999, 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:09:07.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410847548, 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410828866999999, 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410811399000004, 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:09:07.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410795274999998, 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:09:07.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410764168999997, 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:09:07.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410744968000003, 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:09:07.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:07.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:09:07.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.410723926000003, 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:09:07.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:07.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:09:07.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:09:07.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.409968978000002, 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:09:07.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.409942475999998, 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:09:07.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:09:07.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:09:07.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.409924945999997, 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:09:07.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:09:07.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:09:07.671 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.408461785999997, 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:09:07.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.396623138000002, 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:07.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:09:07.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.396606794999997, 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:09:07.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:07.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:09:07.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.683 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:07.684 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:07.684 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:07.685 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:07.686 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:07.686 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:07.687 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:07.693+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:07.786 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:07.794+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:07.814 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:07.814 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:07.815 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:07.815 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:07.824 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:07.886 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:07.895+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:07.951 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:07.951 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:07.951 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:07.961 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:08.002 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:08.011+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:08.088 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:08.096+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:08.128 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:08.128 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.797011350999995, 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.796995444999993, 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.133 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:08.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:08.136 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.796943478000003, 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:08.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:08.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:08.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.140 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.796020003999999, 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:08.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:08.144 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.796003372000001, 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:08.148 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.795301769000005, 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.795285266999997, 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.152 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:08.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.794382401000007, 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.156 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.794374375999993, 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.160 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.793310234000003, 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.164 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.793294265, 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:08.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.171 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.733095826000003, 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:08.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:08.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:08.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.176 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.564071794, 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:08.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.180 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.56405556, 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:08.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:08.184 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.564040339999998, 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:08.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.564012761000001, 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.188 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:08.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.56399446, 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:08.196 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.563978161000001, 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:08.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:08.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.200 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.255575542000001, 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:08.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:08.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:08.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.9063024370000008, 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.205 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:08.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:08.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.209 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:08.210 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:08.211 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:08.218+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:08.311 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:08.319+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:08.374 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:08.374 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:08.384 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:08.428 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:08.436+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:08.506 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:08.507 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:08.507 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:08.507 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:08.507 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:08.512 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:08.520+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:08.517 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:08.613 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:08.621+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.326409808999998, 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.683 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:08.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.326401680000004, 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.687 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.326393656999997, 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.326384744000002, 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:08.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.326375925000001, 2024-04-24T18:09:08.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:08.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.326366792999998, 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.703 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:08.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325546533999997, 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.707 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.710 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325519335000003, 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:08.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:08.714 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325501865, 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:08.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325486525000002, 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:08.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:08.721 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325462447, 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:08.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:08.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325446882000001, 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.726 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:08.729 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325431921000003, 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.730 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.733 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325414981999998, 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:08.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.737 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325398577000001, 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:08.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325382138000002, 2024-04-24T18:09:08.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:08.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325367298000003, 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.745 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.746 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:08.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325350874000002, 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.749 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:08.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325333659000002, 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.753 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:08.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.756 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 38.325316194000003, 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:08.760 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:08.762 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:08.762 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:08.770+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:08.862 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:08.870+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:08.925 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:08.925 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:08.935 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:08.955 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:09:08.979 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:08.988+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:09.064 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:09.072+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:09.072 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:09.072 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:09.072 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:09.072 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:09.072 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:09.083 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:09.165 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:09.173+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:09.249 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:09.249 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:09.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:09.249 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:09.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.249 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.249 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.639826517, 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:09.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:09.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:09.254 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.639802102, 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.255 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:09.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:09.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.639778953, 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.260 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:09.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:09.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:09.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.639698385, 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:09.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:09.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:09.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.639673575, 2024-04-24T18:09:09.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:09.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:09.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:09.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.639653463, 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:09.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.6395828, 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:09.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:09.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.639571027, 2024-04-24T18:09:09.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:09.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:09.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:09.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:09.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.219249198, 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:09.299 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.219202016, 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:09.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.304 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.219176895, 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.305 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:09.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:09.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:09.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:09.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.219124761, 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:09.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.315 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 108.219061297, 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.316 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:09.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:09.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.445562580000001, 2024-04-24T18:09:09.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:09.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.325 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.444456510999998, 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:09.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:09.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:09.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.329 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 37.289284793, 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:09.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:09.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:09.333 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 9.2273308489999994, 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:09.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:09.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 9.2260437660000001, 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.8868676949999994, 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.343 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.345 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 8.8867780009999997, 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.349 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.350 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.350 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:09.351 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:09.351 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:09.359+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:09.451 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:09.459+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:09.517 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:09.517 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:09.517 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:09.517 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:09.527 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:09.552 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:09.560+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:09.651 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:09.651 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:09.651 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:09.651 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:09.651 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:09.653 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:09.662+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:09.661 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:09.688 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:09.754 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:09.763+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:09.828 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:09.828 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:09.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:09.828 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:09.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.828 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.316739575, 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.316718639999999, 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.832 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:09:09.835 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.316688881999999, 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:09:09.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.839 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.313184388, 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:09:09.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:09:09.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.843 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.313160054000001, 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:09:09.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:09:09.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.847 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.313138464, 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.850 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.313122337999999, 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.851 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:09.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.313107445, 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.855 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.312852679000001, 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:09.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:09:09.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:09.862 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.312824883999999, 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:09:09.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:09:09.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.312792619, 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.866 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.312759695, 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.870 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:09:09.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.873 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310844596000001, 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:09:09.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.877 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310834588000001, 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.881 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310821141, 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.882 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310812508, 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.886 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310804011, 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310796375000001, 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310788721, 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:09:09.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.901 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 10.310772171, 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:09.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.905 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.906 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:09.906 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:09.906 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:09.907 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:09.907 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:09.907 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:09.907 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:09.908 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:09.915+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:09.908 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:09.910 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:09.928 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:09.956 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:09.959 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:09:09.978 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:09.980 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:09.981 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:09.981 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:10.034 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:10.042+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:10.062 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:10.062 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:10.073 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:10.109 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:10.117+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:10.209 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:10.217+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:10.241 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:10.241 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:10.251 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:10.311 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:10.319+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:10.411 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:10.419+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:10.421 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:10.431 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:10.515 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:10.523+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:10.601 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:10.611 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:10.613 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:10.622+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:10.714 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:10.722+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:10.780 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:10.780 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:10.790 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:10.815 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:10.824+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:10.916 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:10.924+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:10.949 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:10.949 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:10.950 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:09:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:10.968 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:10.980 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:11.016 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:11.025+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:11.117 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:11.125+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:11.162 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:11.162 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:11.172 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:11.257 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:11.265+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:11.308 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:11.309 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:11.319 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:11.327+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:11.321 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":160,"stamp":"2024-04-24T18:09:10.956197+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1417166654,"num_objects":3815,"num_object_clones":697,"num_object_copies":7630,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3811,"num_whiteouts":2355,"num_read":277729,"num_read_kb":848712,"num_write":26734,"num_write_kb":533501,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":55,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":22249,"ondisk_log_size":22249,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2515708,"kb_used_data":82324,"kb_used_omap":43,"kb_used_meta":2433236,"kb_avail":752459012,"statfs":{"total":773094113280,"available":770518028288,"internally_reserved":0,"allocated":84299776,"data_stored":701338794,"data_compressed":13820471,"data_compressed_allocated":45637632,"data_compressed_original":671756425,"omap_allocated":44665,"internal_metadata":2491634055},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":30428371,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":1,"num_read":13200,"num_read_kb":61092,"num_write":1025,"num_write_kb":30662,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.016474"},"pg_stats":[{"pgid":"4.1f","version":"106'558","reported_seq":6255,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.874505+0000","last_change":"2024-04-24T18:08:56.910909+0000","last_active":"2024-04-24T18:09:06.874505+0000","last_peered":"2024-04-24T18:09:06.874505+0000","last_clean":"2024-04-24T18:09:06.874505+0000","last_became_active":"2024-04-24T18:08:56.900267+0000","last_became_peered":"2024-04-24T18:08:56.900267+0000","last_unstale":"2024-04-24T18:09:06.874505+0000","last_undegraded":"2024-04-24T18:09:06.874505+0000","last_fullsized":"2024-04-24T18:09:06.874505+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":558,"log_dups_size":0,"ondisk_log_size":558,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:07.953531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00047503299999999998,"stat_sum":{"num_bytes":49545826,"num_objects":143,"num_object_clones":26,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":95,"num_read":6897,"num_read_kb":28965,"num_write":556,"num_write_kb":11948,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"107'1199","reported_seq":6722,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.945207+0000","last_change":"2024-04-24T18:09:06.945452+0000","last_active":"2024-04-24T18:09:06.945207+0000","last_peered":"2024-04-24T18:09:06.945207+0000","last_clean":"2024-04-24T18:09:06.945207+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:09:06.945207+0000","last_undegraded":"2024-04-24T18:09:06.945207+0000","last_fullsized":"2024-04-24T18:09:06.945207+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1199,"log_dups_size":0,"ondisk_log_size":1199,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0049260689999999999,"stat_sum":{"num_bytes":45707992,"num_objects":141,"num_object_clones":24,"num_object_copies":282,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":141,"num_whiteouts":89,"num_read":9994,"num_read_kb":25833,"num_write":1793,"num_write_kb":10806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.1d","version":"107'578","reported_seq":5400,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.954152+0000","last_change":"2024-04-24T18:09:06.954399+0000","last_active":"2024-04-24T18:09:06.954152+0000","last_peered":"2024-04-24T18:09:06.954152+0000","last_clean":"2024-04-24T18:09:06.954152+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:09:06.954152+0000","last_undegraded":"2024-04-24T18:09:06.954152+0000","last_fullsized":"2024-04-24T18:09:06.954152+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0067679309999999996,"stat_sum":{"num_bytes":48923298,"num_objects":143,"num_object_clones":26,"num_object_copies":286,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":143,"num_whiteouts":90,"num_read":5817,"num_read_kb":17634,"num_write":552,"num_write_kb":12140,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.1c","version":"107'876","reported_seq":9338,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:07.006167+0000","last_change":"2024-04-24T18:09:06.955278+0000","last_active":"2024-04-24T18:09:07.006167+0000","last_peered":"2024-04-24T18:09:07.006167+0000","last_clean":"2024-04-24T18:09:07.006167+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:09:07.006167+0000","last_undegraded":"2024-04-24T18:09:07.006167+0000","last_fullsized":"2024-04-24T18:09:07.006167+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":876,"log_dups_size":0,"ondisk_log_size":876,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.011018115,"stat_sum":{"num_bytes":39935192,"num_objects":111,"num_object_clones":24,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":64,"num_read":11381,"num_read_kb":22876,"num_write":1195,"num_write_kb":9972,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.1b","version":"107'452","reported_seq":11270,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.972218+0000","last_change":"2024-04-24T18:09:06.972348+0000","last_active":"2024-04-24T18:09:06.972218+0000","last_peered":"2024-04-24T18:09:06.972218+0000","last_clean":"2024-04-24T18:09:06.972218+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:09:06.972218+0000","last_undegraded":"2024-04-24T18:09:06.972218+0000","last_fullsized":"2024-04-24T18:09:06.972218+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":452,"log_dups_size":0,"ondisk_log_size":452,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.017475984,"stat_sum":{"num_bytes":52771434,"num_objects":120,"num_object_clones":28,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":65,"num_read":10912,"num_read_kb":37133,"num_write":456,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.1a","version":"107'748","reported_seq":11289,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:07.051379+0000","last_change":"2024-04-24T18:09:06.955816+0000","last_active":"2024-04-24T18:09:07.051379+0000","last_peered":"2024-04-24T18:09:07.051379+0000","last_clean":"2024-04-24T18:09:07.051379+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:09:07.051379+0000","last_undegraded":"2024-04-24T18:09:07.051379+0000","last_fullsized":"2024-04-24T18:09:07.051379+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":748,"log_dups_size":0,"ondisk_log_size":748,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011284649999999999,"stat_sum":{"num_bytes":41098372,"num_objects":120,"num_object_clones":27,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":68,"num_read":13249,"num_read_kb":29580,"num_write":916,"num_write_kb":12154,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.19","version":"105'621","reported_seq":9241,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.954494+0000","last_change":"2024-04-24T18:09:06.954817+0000","last_active":"2024-04-24T18:09:06.954494+0000","last_peered":"2024-04-24T18:09:06.954494+0000","last_clean":"2024-04-24T18:09:06.954494+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:09:06.954494+0000","last_undegraded":"2024-04-24T18:09:06.954494+0000","last_fullsized":"2024-04-24T18:09:06.954494+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.007054182,"stat_sum":{"num_bytes":41397531,"num_objects":120,"num_object_clones":19,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":77,"num_read":9971,"num_read_kb":31012,"num_write":744,"num_write_kb":15844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.18","version":"107'496","reported_seq":11050,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.955098+0000","last_change":"2024-04-24T18:09:06.955098+0000","last_active":"2024-04-24T18:09:06.955098+0000","last_peered":"2024-04-24T18:09:06.955098+0000","last_clean":"2024-04-24T18:09:06.955098+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:09:06.955098+0000","last_undegraded":"2024-04-24T18:09:06.955098+0000","last_fullsized":"2024-04-24T18:09:06.955098+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":496,"log_dups_size":0,"ondisk_log_size":496,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00046576799999999998,"stat_sum":{"num_bytes":46688097,"num_objects":108,"num_object_clones":19,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":61,"num_read":11823,"num_read_kb":35855,"num_write":516,"num_write_kb":30729,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.17","version":"107'1260","reported_seq":6471,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.955470+0000","last_change":"2024-04-24T18:09:06.955470+0000","last_active":"2024-04-24T18:09:06.955470+0000","last_peered":"2024-04-24T18:09:06.955470+0000","last_clean":"2024-04-24T18:09:06.955470+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:09:06.955470+0000","last_undegraded":"2024-04-24T18:09:06.955470+0000","last_fullsized":"2024-04-24T18:09:06.955470+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1260,"log_dups_size":0,"ondisk_log_size":1260,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0076701390000000003,"stat_sum":{"num_bytes":42133504,"num_objects":116,"num_object_clones":26,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":72,"num_read":10272,"num_read_kb":29137,"num_write":2024,"num_write_kb":19688,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.2","version":"105'1095","reported_seq":4352,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.941200+0000","last_change":"2024-04-24T18:09:06.941898+0000","last_active":"2024-04-24T18:09:06.941200+0000","last_peered":"2024-04-24T18:09:06.941200+0000","last_clean":"2024-04-24T18:09:06.941200+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:09:06.941200+0000","last_undegraded":"2024-04-24T18:09:06.941200+0000","last_fullsized":"2024-04-24T18:09:06.941200+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1095,"log_dups_size":0,"ondisk_log_size":1095,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00099408799999999996,"stat_sum":{"num_bytes":43652270,"num_objects":124,"num_object_clones":20,"num_object_copies":248,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":124,"num_whiteouts":76,"num_read":8065,"num_read_kb":23692,"num_write":1608,"num_write_kb":17576,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"1.7","version":"0'0","reported_seq":72,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-24T18:09:03.943848+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:09:03.943848+0000","last_peered":"2024-04-24T18:09:03.943848+0000","last_clean":"2024-04-24T18:09:03.943848+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:09:03.943848+0000","last_undegraded":"2024-04-24T18:09:03.943848+0000","last_fullsized":"2024-04-24T18:09:03.943848+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":72,"reported_epoch":105,"state":"active+clean","last_fresh":"2024-04-24T18:09:03.943874+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:09:03.943874+0000","last_peered":"2024-04-24T18:09:03.943874+0000","last_clean":"2024-04-24T18:09:03.943874+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:09:03.943874+0000","last_undegraded":"2024-04-24T18:09:03.943874+0000","last_fullsized":"2024-04-24T18:09:03.943874+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"107'635","reported_seq":5869,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:07.051672+0000","last_change":"2024-04-24T18:09:06.948297+0000","last_active":"2024-04-24T18:09:07.051672+0000","last_peered":"2024-04-24T18:09:07.051672+0000","last_clean":"2024-04-24T18:09:07.051672+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:09:07.051672+0000","last_undegraded":"2024-04-24T18:09:07.051672+0000","last_fullsized":"2024-04-24T18:09:07.051672+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":635,"log_dups_size":0,"ondisk_log_size":635,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0073542989999999999,"stat_sum":{"num_bytes":67042023,"num_objects":126,"num_object_clones":24,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":74,"num_read":7097,"num_read_kb":31771,"num_write":665,"num_write_kb":18672,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"3.3","version":"0'0","reported_seq":96,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.943219+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:09:05.943219+0000","last_peered":"2024-04-24T18:09:05.943219+0000","last_clean":"2024-04-24T18:09:05.943219+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:09:05.943219+0000","last_undegraded":"2024-04-24T18:09:05.943219+0000","last_fullsized":"2024-04-24T18:09:05.943219+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":99,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.936944+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:09:05.936944+0000","last_peered":"2024-04-24T18:09:05.936944+0000","last_clean":"2024-04-24T18:09:05.936944+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:09:05.936944+0000","last_undegraded":"2024-04-24T18:09:05.936944+0000","last_fullsized":"2024-04-24T18:09:05.936944+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"107'918","reported_seq":6136,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.952661+0000","last_change":"2024-04-24T18:09:06.953027+0000","last_active":"2024-04-24T18:09:06.952661+0000","last_peered":"2024-04-24T18:09:06.952661+0000","last_clean":"2024-04-24T18:09:06.952661+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:09:06.952661+0000","last_undegraded":"2024-04-24T18:09:06.952661+0000","last_fullsized":"2024-04-24T18:09:06.952661+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":918,"log_dups_size":0,"ondisk_log_size":918,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011520782,"stat_sum":{"num_bytes":32499745,"num_objects":120,"num_object_clones":20,"num_object_copies":240,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":120,"num_whiteouts":84,"num_read":8453,"num_read_kb":13973,"num_write":1182,"num_write_kb":16358,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"3.2","version":"106'3","reported_seq":99,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.877998+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:09:06.877998+0000","last_peered":"2024-04-24T18:09:06.877998+0000","last_clean":"2024-04-24T18:09:06.877998+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:09:06.877998+0000","last_undegraded":"2024-04-24T18:09:06.877998+0000","last_fullsized":"2024-04-24T18:09:06.877998+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":98,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.944815+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:09:05.944815+0000","last_peered":"2024-04-24T18:09:05.944815+0000","last_clean":"2024-04-24T18:09:05.944815+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:09:05.944815+0000","last_undegraded":"2024-04-24T18:09:05.944815+0000","last_fullsized":"2024-04-24T18:09:05.944815+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"105'836","reported_seq":9651,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.209382+0000","last_change":"2024-04-24T18:08:56.912177+0000","last_active":"2024-04-24T18:09:06.209382+0000","last_peered":"2024-04-24T18:09:06.209382+0000","last_clean":"2024-04-24T18:09:06.209382+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:09:06.209382+0000","last_undegraded":"2024-04-24T18:09:06.209382+0000","last_fullsized":"2024-04-24T18:09:06.209382+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":836,"log_dups_size":0,"ondisk_log_size":836,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00031380699999999999,"stat_sum":{"num_bytes":63624539,"num_objects":148,"num_object_clones":24,"num_object_copies":296,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":148,"num_whiteouts":91,"num_read":11598,"num_read_kb":43606,"num_write":1008,"num_write_kb":32702,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":70,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.936504+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:09:05.936504+0000","last_peered":"2024-04-24T18:09:05.936504+0000","last_clean":"2024-04-24T18:09:05.936504+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:09:05.936504+0000","last_undegraded":"2024-04-24T18:09:05.936504+0000","last_fullsized":"2024-04-24T18:09:05.936504+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":98,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.944804+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:09:05.944804+0000","last_peered":"2024-04-24T18:09:05.944804+0000","last_clean":"2024-04-24T18:09:05.944804+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:09:05.944804+0000","last_undegraded":"2024-04-24T18:09:05.944804+0000","last_fullsized":"2024-04-24T18:09:05.944804+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"107'465","reported_seq":4855,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.945579+0000","last_change":"2024-04-24T18:09:06.945579+0000","last_active":"2024-04-24T18:09:06.945579+0000","last_peered":"2024-04-24T18:09:06.945579+0000","last_clean":"2024-04-24T18:09:06.945579+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:09:06.945579+0000","last_undegraded":"2024-04-24T18:09:06.945579+0000","last_fullsized":"2024-04-24T18:09:06.945579+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":465,"log_dups_size":0,"ondisk_log_size":465,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0043005869999999998,"stat_sum":{"num_bytes":31171510,"num_objects":115,"num_object_clones":28,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":66,"num_read":6002,"num_read_kb":29262,"num_write":594,"num_write_kb":15062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"3.1","version":"61'1","reported_seq":97,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.936319+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:09:05.936319+0000","last_peered":"2024-04-24T18:09:05.936319+0000","last_clean":"2024-04-24T18:09:05.936319+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:09:05.936319+0000","last_undegraded":"2024-04-24T18:09:05.936319+0000","last_fullsized":"2024-04-24T18:09:05.936319+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":296,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.944839+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:09:05.944839+0000","last_peered":"2024-04-24T18:09:05.944839+0000","last_clean":"2024-04-24T18:09:05.944839+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:09:05.944839+0000","last_undegraded":"2024-04-24T18:09:05.944839+0000","last_fullsized":"2024-04-24T18:09:05.944839+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":85,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.936640+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:09:05.936640+0000","last_peered":"2024-04-24T18:09:05.936640+0000","last_clean":"2024-04-24T18:09:05.936640+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:09:05.936640+0000","last_undegraded":"2024-04-24T18:09:05.936640+0000","last_fullsized":"2024-04-24T18:09:05.936640+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"107'874","reported_seq":9276,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.946453+0000","last_change":"2024-04-24T18:09:06.946453+0000","last_active":"2024-04-24T18:09:06.946453+0000","last_peered":"2024-04-24T18:09:06.946453+0000","last_clean":"2024-04-24T18:09:06.946453+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:09:06.946453+0000","last_undegraded":"2024-04-24T18:09:06.946453+0000","last_fullsized":"2024-04-24T18:09:06.946453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0057199069999999998,"stat_sum":{"num_bytes":43562429,"num_objects":99,"num_object_clones":16,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":58,"num_read":12192,"num_read_kb":32062,"num_write":1098,"num_write_kb":22130,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.1","version":"106'567","reported_seq":5570,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.955145+0000","last_change":"2024-04-24T18:09:06.955413+0000","last_active":"2024-04-24T18:09:06.955145+0000","last_peered":"2024-04-24T18:09:06.955145+0000","last_clean":"2024-04-24T18:09:06.955145+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:09:06.955145+0000","last_undegraded":"2024-04-24T18:09:06.955145+0000","last_fullsized":"2024-04-24T18:09:06.955145+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":567,"log_dups_size":0,"ondisk_log_size":567,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000683433,"stat_sum":{"num_bytes":48776479,"num_objects":116,"num_object_clones":18,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":83,"num_read":6422,"num_read_kb":29495,"num_write":503,"num_write_kb":23506,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"1.4","version":"0'0","reported_seq":99,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.945196+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:09:05.945196+0000","last_peered":"2024-04-24T18:09:05.945196+0000","last_clean":"2024-04-24T18:09:05.945196+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:09:05.945196+0000","last_undegraded":"2024-04-24T18:09:05.945196+0000","last_fullsized":"2024-04-24T18:09:05.945196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"107'701","reported_seq":5618,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.955648+0000","last_change":"2024-04-24T18:09:06.955648+0000","last_active":"2024-04-24T18:09:06.955648+0000","last_peered":"2024-04-24T18:09:06.955648+0000","last_clean":"2024-04-24T18:09:06.955648+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:09:06.955648+0000","last_undegraded":"2024-04-24T18:09:06.955648+0000","last_fullsized":"2024-04-24T18:09:06.955648+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":701,"log_dups_size":0,"ondisk_log_size":701,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0079940740000000003,"stat_sum":{"num_bytes":41365544,"num_objects":110,"num_object_clones":19,"num_object_copies":220,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":110,"num_whiteouts":62,"num_read":7052,"num_read_kb":18083,"num_write":765,"num_write_kb":9759,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"1.5","version":"0'0","reported_seq":99,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.943204+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:09:05.943204+0000","last_peered":"2024-04-24T18:09:05.943204+0000","last_clean":"2024-04-24T18:09:05.943204+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:09:05.943204+0000","last_undegraded":"2024-04-24T18:09:05.943204+0000","last_fullsized":"2024-04-24T18:09:05.943204+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"107'556","reported_seq":11521,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.948909+0000","last_change":"2024-04-24T18:09:06.948909+0000","last_active":"2024-04-24T18:09:06.948909+0000","last_peered":"2024-04-24T18:09:06.948909+0000","last_clean":"2024-04-24T18:09:06.948909+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:09:06.948909+0000","last_undegraded":"2024-04-24T18:09:06.948909+0000","last_fullsized":"2024-04-24T18:09:06.948909+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":556,"log_dups_size":0,"ondisk_log_size":556,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0035241970000000002,"stat_sum":{"num_bytes":46055361,"num_objects":130,"num_object_clones":25,"num_object_copies":260,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":130,"num_whiteouts":79,"num_read":12129,"num_read_kb":39327,"num_write":571,"num_write_kb":18752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.9","version":"107'573","reported_seq":5890,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.960759+0000","last_change":"2024-04-24T18:09:06.960972+0000","last_active":"2024-04-24T18:09:06.960759+0000","last_peered":"2024-04-24T18:09:06.960759+0000","last_clean":"2024-04-24T18:09:06.960759+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:09:06.960759+0000","last_undegraded":"2024-04-24T18:09:06.960759+0000","last_fullsized":"2024-04-24T18:09:06.960759+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":573,"log_dups_size":0,"ondisk_log_size":573,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.006319369,"stat_sum":{"num_bytes":46300149,"num_objects":117,"num_object_clones":19,"num_object_copies":234,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":117,"num_whiteouts":72,"num_read":6037,"num_read_kb":18012,"num_write":617,"num_write_kb":12791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.a","version":"106'614","reported_seq":3536,"reported_epoch":106,"state":"active+clean","last_fresh":"2024-04-24T18:09:05.995556+0000","last_change":"2024-04-24T18:08:56.908403+0000","last_active":"2024-04-24T18:09:05.995556+0000","last_peered":"2024-04-24T18:09:05.995556+0000","last_clean":"2024-04-24T18:09:05.995556+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:09:05.995556+0000","last_undegraded":"2024-04-24T18:09:05.995556+0000","last_fullsized":"2024-04-24T18:09:05.995556+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":614,"log_dups_size":0,"ondisk_log_size":614,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0003368,"stat_sum":{"num_bytes":26766610,"num_objects":86,"num_object_clones":8,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":60,"num_read":3719,"num_read_kb":11524,"num_write":698,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"107'563","reported_seq":5111,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:07.065150+0000","last_change":"2024-04-24T18:09:06.945964+0000","last_active":"2024-04-24T18:09:07.065150+0000","last_peered":"2024-04-24T18:09:07.065150+0000","last_clean":"2024-04-24T18:09:07.065150+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:09:07.065150+0000","last_undegraded":"2024-04-24T18:09:07.065150+0000","last_fullsized":"2024-04-24T18:09:07.065150+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":563,"log_dups_size":0,"ondisk_log_size":563,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0052086490000000001,"stat_sum":{"num_bytes":25458313,"num_objects":98,"num_object_clones":15,"num_object_copies":196,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":63,"num_read":7608,"num_read_kb":16415,"num_write":611,"num_write_kb":13352,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.c","version":"105'497","reported_seq":6571,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:07.041940+0000","last_change":"2024-04-24T18:09:06.958044+0000","last_active":"2024-04-24T18:09:07.041940+0000","last_peered":"2024-04-24T18:09:07.041940+0000","last_clean":"2024-04-24T18:09:07.041940+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:09:07.041940+0000","last_undegraded":"2024-04-24T18:09:07.041940+0000","last_fullsized":"2024-04-24T18:09:07.041940+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":497,"log_dups_size":0,"ondisk_log_size":497,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.003281066,"stat_sum":{"num_bytes":40085469,"num_objects":97,"num_object_clones":15,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":61,"num_read":7289,"num_read_kb":28453,"num_write":533,"num_write_kb":28659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.d","version":"107'526","reported_seq":8797,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.969496+0000","last_change":"2024-04-24T18:09:06.969496+0000","last_active":"2024-04-24T18:09:06.969496+0000","last_peered":"2024-04-24T18:09:06.969496+0000","last_clean":"2024-04-24T18:09:06.969496+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:09:06.969496+0000","last_undegraded":"2024-04-24T18:09:06.969496+0000","last_fullsized":"2024-04-24T18:09:06.969496+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":526,"log_dups_size":0,"ondisk_log_size":526,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.014687663,"stat_sum":{"num_bytes":47647052,"num_objects":135,"num_object_clones":25,"num_object_copies":270,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":135,"num_whiteouts":85,"num_read":8759,"num_read_kb":23770,"num_write":494,"num_write_kb":9557,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.e","version":"107'491","reported_seq":6688,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.963537+0000","last_change":"2024-04-24T18:09:06.963717+0000","last_active":"2024-04-24T18:09:06.963537+0000","last_peered":"2024-04-24T18:09:06.963537+0000","last_clean":"2024-04-24T18:09:06.963537+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:09:06.963537+0000","last_undegraded":"2024-04-24T18:09:06.963537+0000","last_fullsized":"2024-04-24T18:09:06.963537+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":491,"log_dups_size":0,"ondisk_log_size":491,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0089058420000000006,"stat_sum":{"num_bytes":43117226,"num_objects":112,"num_object_clones":24,"num_object_copies":224,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":112,"num_whiteouts":63,"num_read":6598,"num_read_kb":16216,"num_write":519,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.f","version":"107'725","reported_seq":5734,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.946312+0000","last_change":"2024-04-24T18:09:06.946491+0000","last_active":"2024-04-24T18:09:06.946312+0000","last_peered":"2024-04-24T18:09:06.946312+0000","last_clean":"2024-04-24T18:09:06.946312+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:09:06.946312+0000","last_undegraded":"2024-04-24T18:09:06.946312+0000","last_fullsized":"2024-04-24T18:09:06.946312+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":725,"log_dups_size":0,"ondisk_log_size":725,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0053536449999999998,"stat_sum":{"num_bytes":37383530,"num_objects":122,"num_object_clones":18,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":73,"num_read":7697,"num_read_kb":20327,"num_write":830,"num_write_kb":8608,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.10","version":"106'547","reported_seq":6871,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.947685+0000","last_change":"2024-04-24T18:09:06.948003+0000","last_active":"2024-04-24T18:09:06.947685+0000","last_peered":"2024-04-24T18:09:06.947685+0000","last_clean":"2024-04-24T18:09:06.947685+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:09:06.947685+0000","last_undegraded":"2024-04-24T18:09:06.947685+0000","last_fullsized":"2024-04-24T18:09:06.947685+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":547,"log_dups_size":0,"ondisk_log_size":547,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054469999999999996,"stat_sum":{"num_bytes":29538394,"num_objects":128,"num_object_clones":24,"num_object_copies":256,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":128,"num_whiteouts":79,"num_read":7626,"num_read_kb":18105,"num_write":604,"num_write_kb":14126,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.11","version":"103'658","reported_seq":8718,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.947738+0000","last_change":"2024-04-24T18:09:06.948113+0000","last_active":"2024-04-24T18:09:06.947738+0000","last_peered":"2024-04-24T18:09:06.947738+0000","last_clean":"2024-04-24T18:09:06.947738+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:09:06.947738+0000","last_undegraded":"2024-04-24T18:09:06.947738+0000","last_fullsized":"2024-04-24T18:09:06.947738+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":658,"log_dups_size":0,"ondisk_log_size":658,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00052217599999999996,"stat_sum":{"num_bytes":95480404,"num_objects":132,"num_object_clones":27,"num_object_copies":264,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":132,"num_whiteouts":84,"num_read":9360,"num_read_kb":48621,"num_write":815,"num_write_kb":30856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.12","version":"103'543","reported_seq":5763,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.941263+0000","last_change":"2024-04-24T18:09:06.941965+0000","last_active":"2024-04-24T18:09:06.941263+0000","last_peered":"2024-04-24T18:09:06.941263+0000","last_clean":"2024-04-24T18:09:06.941263+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:09:06.941263+0000","last_undegraded":"2024-04-24T18:09:06.941263+0000","last_fullsized":"2024-04-24T18:09:06.941263+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":543,"log_dups_size":0,"ondisk_log_size":543,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00096181299999999995,"stat_sum":{"num_bytes":26740303,"num_objects":122,"num_object_clones":16,"num_object_copies":244,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":122,"num_whiteouts":82,"num_read":9434,"num_read_kb":30904,"num_write":514,"num_write_kb":8693,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.13","version":"107'886","reported_seq":8764,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:07.004681+0000","last_change":"2024-04-24T18:09:06.969670+0000","last_active":"2024-04-24T18:09:07.004681+0000","last_peered":"2024-04-24T18:09:07.004681+0000","last_clean":"2024-04-24T18:09:07.004681+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:09:07.004681+0000","last_undegraded":"2024-04-24T18:09:07.004681+0000","last_fullsized":"2024-04-24T18:09:07.004681+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":886,"log_dups_size":0,"ondisk_log_size":886,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.014686286999999999,"stat_sum":{"num_bytes":43298624,"num_objects":111,"num_object_clones":26,"num_object_copies":222,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":111,"num_whiteouts":67,"num_read":10357,"num_read_kb":22657,"num_write":1260,"num_write_kb":26655,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.14","version":"107'540","reported_seq":7246,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.958287+0000","last_change":"2024-04-24T18:09:06.958287+0000","last_active":"2024-04-24T18:09:06.958287+0000","last_peered":"2024-04-24T18:09:06.958287+0000","last_clean":"2024-04-24T18:09:06.958287+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:09:06.958287+0000","last_undegraded":"2024-04-24T18:09:06.958287+0000","last_fullsized":"2024-04-24T18:09:06.958287+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":540,"log_dups_size":0,"ondisk_log_size":540,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.01046765,"stat_sum":{"num_bytes":49500631,"num_objects":97,"num_object_clones":20,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":57,"num_read":8986,"num_read_kb":24949,"num_write":484,"num_write_kb":9828,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.15","version":"107'499","reported_seq":5329,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.947840+0000","last_change":"2024-04-24T18:09:06.947840+0000","last_active":"2024-04-24T18:09:06.947840+0000","last_peered":"2024-04-24T18:09:06.947840+0000","last_clean":"2024-04-24T18:09:06.947840+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:09:06.947840+0000","last_undegraded":"2024-04-24T18:09:06.947840+0000","last_fullsized":"2024-04-24T18:09:06.947840+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":499,"log_dups_size":0,"ondisk_log_size":499,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0022426630000000002,"stat_sum":{"num_bytes":20515746,"num_objects":108,"num_object_clones":20,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":69,"num_read":6853,"num_read_kb":16891,"num_write":431,"num_write_kb":9134,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"37","length":"1"}]},{"pgid":"4.16","version":"105'1031","reported_seq":6059,"reported_epoch":107,"state":"active+clean","last_fresh":"2024-04-24T18:09:06.941024+0000","last_change":"2024-04-24T18:09:06.941353+0000","last_active":"2024-04-24T18:09:06.941024+0000","last_peered":"2024-04-24T18:09:06.941024+0000","last_clean":"2024-04-24T18:09:06.941024+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:09:06.941024+0000","last_undegraded":"2024-04-24T18:09:06.941024+0000","last_fullsized":"2024-04-24T18:09:06.941024+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1031,"log_dups_size":0,"ondisk_log_size":1031,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076053000000000004,"stat_sum":{"num_bytes":58792604,"num_objects":134,"num_object_clones":27,"num_object_copies":268,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":134,"num_whiteouts":83,"num_read":7964,"num_read_kb":32472,"num_write":1418,"num_write_kb":16900,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"37","length":"1"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":1416576201,"num_objects":3809,"num_object_clones":697,"num_object_copies":7618,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3809,"num_whiteouts":2352,"num_read":277613,"num_read_kb":848612,"num_write":26574,"num_write_kb":531084,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":55,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":72982528,"data_stored":683892443,"data_compressed":13580926,"data_compressed_allocated":44724224,"data_compressed_original":659554908,"omap_allocated":9738,"internal_metadata":0},"log_size":22128,"ondisk_log_size":22128,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":85,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574873,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310204,"kb_used_data":8748,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94061636,"statfs":{"total":96636764160,"available":96319115264,"internally_reserved":0,"allocated":8957952,"data_stored":75867391,"data_compressed":1201158,"data_compressed_allocated":4861952,"data_compressed_original":72942659,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.252,"5min":0.252,"15min":0.252},"max":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.38800000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.69899999999999995,"5min":0.69899999999999995,"15min":0.69899999999999995},"last":0.44400000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48999999999999999},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.377},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"last":0.35699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.68799999999999994,"5min":0.68799999999999994,"15min":0.68799999999999994},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"last":0.28199999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.318},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.022,"5min":1.022,"15min":1.022},"last":0.41899999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.39800000000000002}]}]},{"osd":6,"up_from":95,"seq":408021893124,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311316,"kb_used_data":8584,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94060524,"statfs":{"total":96636764160,"available":96317976576,"internally_reserved":0,"allocated":8790016,"data_stored":72223892,"data_compressed":1690818,"data_compressed_allocated":5148672,"data_compressed_original":69512885,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.23699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29499999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.315}]}]},{"osd":1,"up_from":13,"seq":55834574872,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":328296,"kb_used_data":14556,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94043544,"statfs":{"total":96636764160,"available":96300589056,"internally_reserved":0,"allocated":14905344,"data_stored":105838683,"data_compressed":2103993,"data_compressed_allocated":7159808,"data_compressed_original":99786764,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":4.9249999999999998,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.32000000000000001},{"interface":"front","average":{"1min":1.244,"5min":1.244,"15min":1.244},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":14.15,"5min":14.15,"15min":14.15},"last":0.30199999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.306,"5min":1.306,"15min":1.306},"last":0.35899999999999999},{"interface":"front","average":{"1min":1.302,"5min":1.302,"15min":1.302},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":14.118,"5min":14.118,"15min":14.118},"last":0.37}]},{"osd":3,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2629999999999999,"5min":1.2629999999999999,"15min":1.2629999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":14.089,"5min":14.089,"15min":14.089},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":5.1139999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.39400000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.41699999999999998},{"interface":"front","average":{"1min":1.3819999999999999,"5min":1.3819999999999999,"15min":1.3819999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":14.007999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.503}]},{"osd":5,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":0.80100000000000005,"5min":0.80100000000000005,"15min":0.80100000000000005},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":5.0119999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68600000000000005},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"max":{"1min":1.49,"5min":1.49,"15min":1.49},"last":0.46300000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.53700000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.4039999999999999,"5min":1.4039999999999999,"15min":1.4039999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":14.051,"5min":14.051,"15min":14.051},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":5.0839999999999996,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.47799999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574873,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304540,"kb_used_data":7180,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94067300,"statfs":{"total":96636764160,"available":96324915200,"internally_reserved":0,"allocated":7352320,"data_stored":64304019,"data_compressed":1174382,"data_compressed_allocated":4128768,"data_compressed_original":62063500,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.375,"5min":1.375,"15min":1.375},"last":0.45500000000000002},{"interface":"front","average":{"1min":1.8300000000000001,"5min":1.8300000000000001,"15min":1.8300000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":26.936,"5min":26.936,"15min":26.936},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.262,"5min":1.262,"15min":1.262},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":11.93,"5min":11.93,"15min":11.93},"last":0.495},{"interface":"front","average":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.075,"5min":1.075,"15min":1.075},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":14.371,"5min":14.371,"15min":14.371},"last":0.35999999999999999},{"interface":"front","average":{"1min":1.028,"5min":1.028,"15min":1.028},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":11.973000000000001,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.52900000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":1.208,"5min":1.208,"15min":1.208},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":14.414,"5min":14.414,"15min":14.414},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":1.329,"5min":1.329,"15min":1.329},"last":0.60499999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":6.0010000000000003,"5min":6.0010000000000003,"15min":6.0010000000000003},"last":0.34100000000000003},{"interface":"front","average":{"1min":1.214,"5min":1.214,"15min":1.214},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":14.452,"5min":14.452,"15min":14.452},"last":0.68899999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.82099999999999995,"15min":0.82099999999999995},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.0359999999999996,"5min":6.0359999999999996,"15min":6.0359999999999996},"last":0.70799999999999996},{"interface":"front","average":{"1min":1.2290000000000001,"5min":1.2290000000000001,"15min":1.2290000000000001},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":14.486000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.73499999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574872,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314156,"kb_used_data":4512,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94057684,"statfs":{"total":96636764160,"available":96315068416,"internally_reserved":0,"allocated":4620288,"data_stored":28503394,"data_compressed":572358,"data_compressed_allocated":1982464,"data_compressed_original":26719933,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"min":{"1min":0.20399999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":1.5589999999999999,"5min":1.5589999999999999,"15min":1.5589999999999999},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.63}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.88100000000000001,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.044,"5min":1.044,"15min":1.044},"last":0.52200000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.65600000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.56200000000000006,"15min":0.56200000000000006},"min":{"1min":0.32500000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.93000000000000005,"5min":0.93000000000000005,"15min":0.93000000000000005},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.155,"5min":1.155,"15min":1.155},"last":0.67400000000000004}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.625,"15min":0.625},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.61599999999999999,"15min":0.61599999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":0.94799999999999995,"15min":0.94799999999999995},"last":0.45600000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574873,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":325536,"kb_used_data":15892,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94046304,"statfs":{"total":96636764160,"available":96303415296,"internally_reserved":0,"allocated":16273408,"data_stored":146997862,"data_compressed":2473371,"data_compressed_allocated":9056256,"data_compressed_original":141167482,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.23499999999999999},"max":{"1min":0.73799999999999999,"5min":0.73799999999999999,"15min":0.73799999999999999},"last":0.223},{"interface":"front","average":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"last":0.35999999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":1.792,"5min":1.792,"15min":1.792},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.307}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.375,"15min":0.375},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.438,"5min":0.438,"15min":0.438},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.625,"5min":0.625,"15min":0.625},"last":0.34699999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55800000000000005,"15min":0.55800000000000005},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81399999999999995,"5min":0.81399999999999995,"15min":0.81399999999999995},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.72499999999999998,"5min":0.72499999999999998,"15min":0.72499999999999998},"last":0.45400000000000001}]},{"osd":5,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.441},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.63500000000000001}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.58099999999999996,"5min":0.58099999999999996,"15min":0.58099999999999996},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.755,"5min":0.755,"15min":0.755},"last":3.8180000000000001},{"interface":"front","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.48099999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574873,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":314128,"kb_used_data":12680,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94057712,"statfs":{"total":96636764160,"available":96315097088,"internally_reserved":0,"allocated":12984320,"data_stored":123329874,"data_compressed":2758406,"data_compressed_allocated":7643136,"data_compressed_original":118960212,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.40799999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.41799999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.36799999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.35799999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.42799999999999999},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.49199999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.30599999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.33200000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.32300000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574873,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307532,"kb_used_data":10172,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94064308,"statfs":{"total":96636764160,"available":96321851392,"internally_reserved":0,"allocated":10416128,"data_stored":84273679,"data_compressed":1845985,"data_compressed_allocated":5656576,"data_compressed_original":80602990,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.189,"5min":0.189,"15min":0.189},"max":{"1min":0.67000000000000004,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.222,"5min":0.222,"15min":0.222},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.51200000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.443,"5min":0.443,"15min":0.443},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.84899999999999998,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.296,"5min":1.296,"15min":1.296},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":1.377,"5min":1.377,"15min":1.377},"last":0.42199999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.899,"5min":1.899,"15min":1.899},"last":0.438},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.66100000000000003,"15min":0.66100000000000003},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.881,"5min":1.881,"15min":1.881},"last":0.45200000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.52000000000000002,"15min":0.52000000000000002},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.25},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.0640000000000001,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.372}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.307},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.28100000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.38200000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":6078464,"data_stored":63672656,"data_compressed":1174382,"data_compressed_allocated":4128768,"data_compressed_original":62063500,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":13262848,"data_stored":101611616,"data_compressed":2035110,"data_compressed_allocated":6909952,"data_compressed_original":96301298,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":3235840,"data_stored":27004482,"data_compressed":568185,"data_compressed_allocated":1941504,"data_compressed_original":25924937,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":14778368,"data_stored":143736746,"data_compressed":2416058,"data_compressed_allocated":8843264,"data_compressed_original":138537850,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":11710464,"data_stored":122698511,"data_compressed":2758406,"data_compressed_allocated":7643136,"data_compressed_original":118960212,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":9138176,"data_stored":83642231,"data_compressed":1845985,"data_compressed_allocated":5656576,"data_compressed_original":80602990,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":7528448,"data_stored":71599133,"data_compressed":1690818,"data_compressed_allocated":5148672,"data_compressed_original":69512885,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":7249920,"data_stored":69927068,"data_compressed":1091982,"data_compressed_allocated":4452352,"data_compressed_original":67651236,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:11.343 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:11.344 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:11.354 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:11.420 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:11.428+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:11.520 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:11.529+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:11.523 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:11.523 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:11.534 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:11.621 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:11.630+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:11.662 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:11.662 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:11.672 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:11.722 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:11.730+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:11.801 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:11.802 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:11.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:11.823 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:11.831+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:11.924 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:11.932+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:11.972 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:09:11.989 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:11.989 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:11.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:12.003 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:12.045 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:12.053+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:12.125 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:12.134+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:12.156 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:12.156 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:12.160 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:12.168 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:12.179 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:12.197 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:12.217 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:12.226 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:12.234+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:12.236 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:12.254 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:12.273 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:12.292 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:12.293 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:12.293 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:12.293 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:12.293 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:12.293 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:12.293 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:12.294 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:12.294 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:12.327 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:12.336+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:12.335 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:12.335 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:12.335 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:12.335 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:12.335 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:12.345 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:12.430 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:12.439+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:12.511 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.511 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.179440088000007, 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.179424182000005, 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.516 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.520 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.179372215000001, 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:12.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.178448740999997, 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.526 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:12.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.178432108999999, 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:12.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.535 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.177730506000003, 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:12.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.177714003999995, 2024-04-24T18:09:12.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:12.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.176811138000005, 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.545 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:12.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.549 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:12.550 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.176803113000005, 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:12.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.175738971000001, 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.559 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 83.175723001999998, 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:12.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.115524563000001, 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:12.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:12.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.946500530999998, 2024-04-24T18:09:12.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:12.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.946484296999998, 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.576 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:12.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:12.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.946469077000003, 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:12.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.585 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.946441497999999, 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.586 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:12.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.946423197000001, 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 35.946406897999999, 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.595 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:12.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.638004279, 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.600 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:12.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.288731174, 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:12.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.609 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:12.610 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:12.610 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:12.611 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:12.612 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:12.619+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:12.711 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:12.719+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:12.776 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:12.776 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:12.776 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:12.786 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:12.830 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:12.839+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:12.909 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:12.909 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:12.909 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:12.912 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:12.921+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:12.919 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:12.965 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:09:13.013 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:13.021+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:13.092 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:13.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735985931000002, 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735977802000001, 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:13.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735969779000001, 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.104 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735960865999999, 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.105 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:13.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:13.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735952046999998, 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:13.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:13.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:13.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735942915000003, 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.113 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:13.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.116 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735122656000001, 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735095457, 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.124 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735077986999997, 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735062646999999, 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:13.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:13.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:13.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.132 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735038568999997, 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:13.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:13.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.136 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735023003999999, 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.735008043000001, 2024-04-24T18:09:13.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:13.144 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.734991104000002, 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.734974698999999, 2024-04-24T18:09:13.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:13.152 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.734958259999999, 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:13.156 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.73494342, 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:13.160 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.734926995999999, 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:13.164 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.734909780999999, 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:13.168 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.734892316, 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:13.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.172 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:13.174 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:13.174 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:13.182+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:13.274 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:13.283+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:13.340 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:13.340 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:13.340 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:13.350 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:13.391 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:13.400+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:13.476 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:13.484+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:13.514 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:13.514 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:13.514 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:13.524 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:13.577 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:13.585+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.039801051, 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.649 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:13.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.039776636, 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.654 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:13.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:13.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.039753487, 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.659 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:13.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:13.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.039672919, 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:13.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:13.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.039648109, 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.669 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:13.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.039627997, 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:13.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.03955733399999, 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:13.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 113.039545561, 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:13.683 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:13.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:13.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:13.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.61922373199999, 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.688 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.61917655000001, 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.693 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:13.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:13.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.619151429, 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.698 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:13.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.619099295, 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.703 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:13.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:13.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.61903583100001, 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.708 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:13.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.845537113999995, 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:13.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.844431044999993, 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:13.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:13.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:13.720 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 41.689259327000002, 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.627305382999999, 2024-04-24T18:09:13.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.6260183, 2024-04-24T18:09:13.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:13.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:13.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:13.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.286842228999999, 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:13.732 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:13.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 13.286752535, 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:13.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:13.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:13.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:13.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:13.740 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:13.741 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:13.741 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:13.749+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:13.842 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:13.850+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:13.906 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:13.906 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:13.906 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:13.906 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:13.916 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:13.942 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:13.950+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:14.043 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:14.051+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:14.087 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:14.087 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:14.087 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:14.088 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:14.088 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:14.098 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:14.144 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:14.152+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:14.223 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:14.223 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:14.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:14.223 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:14.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.712112971, 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.712092036, 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:09:14.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:09:14.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.231 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.712062277999999, 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:09:14.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.708557784, 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:09:14.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:09:14.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.708533450000001, 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.70851186, 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:09:14.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:09:14.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.246 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.708495734, 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:14.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:09:14.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:09:14.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.250 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.708480841, 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:09:14.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.708226075000001, 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.254 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:09:14.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.257 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.70819828, 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:09:14.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:09:14.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:09:14.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:09:14.261 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.708166015, 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.262 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:09:14.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:09:14.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.708133091000001, 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.266 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706217991999999, 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.270 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706207984000001, 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:09:14.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:09:14.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706194537, 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.278 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:09:14.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:09:14.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706185904, 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:09:14.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706177407, 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:09:14.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:09:14.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.289 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706169771000001, 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:14.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:09:14.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706162117, 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:09:14.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 14.706145567, 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:14.297 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:14.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:14.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:14.301 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:14.302 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:14.303 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:14.310+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:14.305 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:14.403 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:14.411+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:14.452 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:14.452 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:14.452 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:14.452 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:14.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:14.474 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:14.498 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:14.504 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:14.512+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:14.514 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:14.533 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:14.552 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:14.570 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:14.589 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:14.604 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:14.605 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:14.605 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:14.612+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:14.608 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:14.609 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:14.610 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:14.616 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:14.706 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:14.714+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:14.789 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:14.789 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:14.799 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:14.806 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:14.815+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:14.907 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:14.915+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:14.929 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:14.929 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:14.939 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:14.985 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:09:15.023 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:15.031+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:15.110 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:15.117+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:15.124 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:15.124 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:15.135 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:15.221 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:15.229+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:15.306 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:15.306 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:15.311 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:15.319+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:15.317 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:15.413 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:15.420+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:15.486 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:15.486 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:15.496 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:15.513 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:15.522+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:15.615 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:15.623+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:15.664 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:15.664 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:15.674 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:15.716 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:15.724+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:15.802 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:15.802 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:15.812 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:15.817 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:15.825+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:15.917 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:15.926+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:15.941 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:15.941 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:15.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:15.981 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:09:16.037 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:16.045+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:16.098 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:16.098 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:16.109 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:16.119 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:16.127+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:16.220 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:16.228+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:16.278 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:16.278 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:16.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:16.325 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:16.325 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-24T18:09:16.333 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:16.341+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:16.420 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:16.420 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:16.421 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:16.430+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:16.430 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:16.522 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:16.531+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:16.596 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:16.596 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:16.605 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:16.622 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:16.647 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:16.655+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:16.724 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:16.732+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:16.768 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:16.769 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:16.769 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:16.779 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:16.787 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:16.804 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:16.824 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:16.824 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:16.832+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:16.842 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:16.860 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:16.879 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:16.898 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:16.899 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:16.900 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:16.925 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:16.933+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:16.945 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:16.946 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:16.946 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:16.946 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:16.956 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:16.982 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.0 to 1 (10000) 2024-04-24T18:09:16.996 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-24T18:09:17.040 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:17.049+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:17.127 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:17.135+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:17.128 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:17.128 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:17.128 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:17.128 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:17.128 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:17.139 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:17.228 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:17.237+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:17.307 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.950665538000003, 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.308 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:17.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:17.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:17.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.312 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.950657409000002, 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.313 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.950649386000002, 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.318 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:17.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.322 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.950640473, 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.323 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:17.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.327 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.950631653999999, 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.328 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.330 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.332 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.950622522000003, 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.333 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.337 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949802263000002, 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.338 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.342 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949775064000001, 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:17.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:17.345 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949757593999998, 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.348 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:17.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.351 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:17.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:17.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.351 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949742254, 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.354 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:17.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949718175999998, 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:17.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949702610999999, 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:17.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:17.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949687650000001, 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:17.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:17.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949670711000003, 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.372 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:17.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:17.376 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949654305999999, 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:17.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.381 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949637867, 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:17.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949623027000001, 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.386 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:17.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:17.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.390 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949606602999999, 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.391 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:17.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:17.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.396 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949589388, 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:17.400 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 46.949571923000001, 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:17.401 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.405 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:17.407 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:17.407 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:17.414+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:17.508 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:17.516+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:17.572 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:17.572 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:17.572 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:17.572 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:17.582 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:17.626 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:17.635+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:17.706 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:17.706 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:17.706 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:17.706 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:17.706 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:17.708 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:17.717+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:17.715 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:17.809 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:17.817+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.27175524099999, 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.881 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:17.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.886 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.271730826, 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.887 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.888 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.889 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.890 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.27170767699999, 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:17.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:17.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:17.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:17.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:17.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.27162710899999, 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.897 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:17.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.902 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.27160229899999, 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:17.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:17.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.27158218700001, 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:17.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:17.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:17.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.271511524, 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.27149975099999, 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.919 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:17.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:17.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 116.85117792200001, 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:17.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:17.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 116.85113074, 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.929 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 116.85110561899999, 2024-04-24T18:09:17.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 116.85105348499999, 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 116.850990021, 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.945 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:17.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:17.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:17.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:17.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.950 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.077491304000006, 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:17.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:17.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:17.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.954 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.076385235000004, 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:17.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:17.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:17.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 45.921213516999998, 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:17.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:17.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:17.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.962 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.859259572999999, 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:17.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:17.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.857972490000002, 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:17.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:17.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:17.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:17.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.518796419000001, 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:17.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:17.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 17.518706725000001, 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:17.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:17.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:17.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:17.979 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:17.980 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:17.981 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:17.989+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:17.995 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:09:18.081 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:18.089+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:18.145 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:18.145 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:18.145 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:18.145 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:18.155 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:18.182 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:18.189+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:18.278 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:18.278 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:18.278 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:18.278 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:18.278 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:18.282 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:18.291+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:18.288 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:18.304 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.2 to 1 (10000) 2024-04-24T18:09:18.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-24T18:09:18.405 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:18.413+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.946353781999999, 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:18.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:09:18.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:09:18.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.946332847000001, 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:09:18.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:09:18.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:09:18.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.946303089000001, 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:09:18.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:18.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:09:18.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:09:18.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942798594999999, 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:09:18.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:18.474 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942774261, 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:18.478 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942752671000001, 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:18.482 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942736544999999, 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:09:18.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.486 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942721651999999, 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:18.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:09:18.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:09:18.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942466885999998, 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:09:18.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942439091000001, 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:09:18.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942406825999999, 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.499 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:09:18.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:09:18.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.942373902, 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.503 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:18.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:09:18.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:09:18.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940458802999999, 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.507 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940448795000002, 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.511 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940435348000001, 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.515 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:09:18.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:09:18.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940426715000001, 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.519 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:18.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:09:18.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:09:18.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940418218000001, 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:18.527 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940410581999998, 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:09:18.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.531 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940402928000001, 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:18.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:09:18.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 18.940386377999999, 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.539 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.540 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.540 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:18.540 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:18.540 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:18.541 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:18.542 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:18.549+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:18.642 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:18.650+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:18.670 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:18.670 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:18.670 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:18.680 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:18.742 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:18.750+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:18.808 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:18.808 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:18.808 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:18.818 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:18.844 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:18.852+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:18.911 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:18.945 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:18.953+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:18.990 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.990 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.292042797999997, 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:18.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:09:18.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:09:18.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:18.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:18.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 82.379567992000005, 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:18.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:18.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:09:18.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:09:19.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.952099468999997, 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.004 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:19.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:09:19.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:19.010 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.952078184000001, 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:09:19.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.952060821000003, 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.016 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:09:19.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.952015428000003, 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.020 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:09:19.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951987926999998, 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:09:19.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:19.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.027 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951969366999997, 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.028 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:09:19.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.031 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951945479000003, 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:09:19.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951927585, 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:09:19.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951892942999997, 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:19.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951798414999999, 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:09:19.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:09:19.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.047 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951779733999999, 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:19.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:09:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951724255000002, 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:09:19.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951689868999999, 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951667909999998, 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.064 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951650827000002, 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:09:19.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951634366999997, 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:09:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951560452000002, 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:09:19.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.076 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 42.951507929000002, 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:19.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:19.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:19.080 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:19.082 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:19.082 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:19.083 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:19.083 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:19.090+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:19.092 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:19.111 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:19.149 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:19.168 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:19.182 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:19.190+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:19.186 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:19.206 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:19.207 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:19.208 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:19.225 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:19.226 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:19.236 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:19.283 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:19.291+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.3 to 1 (10000) 2024-04-24T18:09:19.320 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:09:19.335 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-24T18:09:19.420 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:19.428+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:19.425 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:19.425 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:19.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:19.484 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:19.492+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:19.584 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:19.592+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:19.622 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:19.622 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:19.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:19.686 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:19.694+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:19.786 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:19.795+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:19.800 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:19.800 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:19.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:19.887 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:19.895+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:19.939 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:19.939 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:19.949 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:19.988 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:19.996+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:20.089 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:20.097+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:20.118 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:20.118 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:20.128 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:20.189 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:20.197+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:20.256 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:20.256 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:20.266 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:20.290 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:20.298+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:20.322 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.4 to 1 (10000) 2024-04-24T18:09:20.334 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-24T18:09:20.419 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:20.427+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:20.453 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:20.453 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:20.464 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:20.491 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:20.500+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:20.592 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:20.600+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:20.635 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:20.635 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:20.646 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:20.693 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:20.700+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:20.783 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:20.784 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:20.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:20.794 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:20.802+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:20.894 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:20.903+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:20.960 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:20.960 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:20.969 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:21.011 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:21.019+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:21.096 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:21.105+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:21.134 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:21.134 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:21.144 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:21.198 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:21.206+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:21.219 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:21.268 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:21.269 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:21.278 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:21.298 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:21.306+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:21.327 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.5 to 1 (10000) 2024-04-24T18:09:21.337 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:09:21.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-24T18:09:21.386 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:21.410 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:21.410 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:21.418+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:21.413 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:21.413 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:21.425 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:21.442 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:21.464 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:21.492 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:21.500 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:21.508+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:21.513 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:21.532 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:21.555 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:21.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.199075327000003, 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:21.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.199067198000002, 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.560 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:21.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:21.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:21.563 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.199059175000002, 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.564 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:21.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:21.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:21.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.199050262, 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.568 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.199041442999999, 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.572 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.576 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.199032311000003, 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:21.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:21.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.580 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198212052000002, 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:21.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.584 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198184853000001, 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:21.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:21.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:21.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198167382999998, 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.589 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198152043, 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.593 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:21.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:21.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198127964999998, 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:21.597 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.600 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198112399999999, 2024-04-24T18:09:21.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:21.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.607 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198097439000001, 2024-04-24T18:09:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.609 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.612 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:21.613 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:21.614 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198080500000003, 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:21.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:21.616 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:21.617 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.618 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198064094999999, 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.619 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.620 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.621 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.622 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:21.623 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198047656, 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.624 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.625 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.626 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.627 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198032816000001, 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.628 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.629 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:21.630 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:21.631 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.198016392, 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.632 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.633 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.634 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.635 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.197999177, 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.636 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.637 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.638 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.639 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.197981712000001, 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:21.640 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:21.641 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:21.642 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:21.643 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:21.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:21.645 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:21.646 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:21.647 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:21.649 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:21.650 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:21.655+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:21.650 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:21.651 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:21.651 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:21.651 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:21.651 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:21.651 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:21.651 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:21.652 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:21.748 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:21.756+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:21.814 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:21.814 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:21.814 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:21.824 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:21.849 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:21.858+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:21.948 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:21.948 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:21.948 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:21.949 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:21.958+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:21.958 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:22.050 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:22.059+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:22.087 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.47459031299999, 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:22.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:22.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:22.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:22.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.47456589799999, 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:22.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:22.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.47454274899999, 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:22.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.47446218100001, 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:22.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:22.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:22.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.112 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.47443737099999, 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:22.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:22.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:22.117 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.47441725900001, 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:22.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:22.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:22.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:22.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.474346596, 2024-04-24T18:09:22.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.128 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.47433482300001, 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:22.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:22.134 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.054012994, 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:22.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:22.139 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.053965812, 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.140 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:22.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.145 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.05394069099999, 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:22.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:22.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.152 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.05388855699999, 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:22.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:22.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.053825093, 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.159 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:22.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:22.165 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.280326376000005, 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:22.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:22.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.279220307000003, 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:22.170 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:22.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 50.124048588999997, 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.175 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:22.179 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.062094644999998, 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 22.060807562000001, 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:22.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:22.189 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 21.721631491, 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:22.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 21.721541797, 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:22.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.199 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.199 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:22.201 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:22.201 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:22.209+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:22.303 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:22.310+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:22.304 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:09:22.304 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.6 to 1 (10000) 2024-04-24T18:09:22.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-24T18:09:22.373 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:22.373 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:22.373 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:22.386 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:22.430 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:22.439+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:22.503 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:22.511+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:22.509 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:22.510 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:22.510 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:22.510 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:22.510 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:22.520 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:22.604 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:22.612+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:22.685 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:22.685 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:22.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:22.685 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.174108174000001, 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.689 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.174087238999999, 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:09:22.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:09:22.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.693 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.694 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.694 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:22.694 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.174057480999998, 2024-04-24T18:09:22.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:09:22.697 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.697 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.697 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.697 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:09:22.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:09:22.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170552987000001, 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:22.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:09:22.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170528653000002, 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:22.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:09:22.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:09:22.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170507062999999, 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:09:22.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:22.713 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170490937, 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.717 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170476044000001, 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:22.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.721 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170221278, 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.722 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170193482999998, 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.726 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:09:22.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170161218000001, 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:09:22.730 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:22.734 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.170128294000001, 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.738 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168213195, 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.742 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168203187, 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.746 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168189739999999, 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.750 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168181106999999, 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:09:22.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168172609999999, 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:09:22.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:09:22.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:09:22.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168164974, 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168157319999999, 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.762 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 23.168140770000001, 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:22.766 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:22.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:09:22.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:09:22.770 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.770 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.770 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.770 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:22.770 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:22.770 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:22.771 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:22.780+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:22.772 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:22.872 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:22.881+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:22.900 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:22.900 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:22.901 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:22.910 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:22.974 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:22.983+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:23.038 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:23.038 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:23.048 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:23.092 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:23.100+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:23.175 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:23.183+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:23.177 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:23.177 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:23.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:23.177 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.479675842000006, 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.178 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 86.567201036, 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.183 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139732512999998, 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:09:23.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:23.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:09:23.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:09:23.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139711228000003, 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:09:23.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139693864999998, 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:09:23.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:09:23.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139648471999998, 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:09:23.199 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:23.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:09:23.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139620970999999, 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.203 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139602410999998, 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.207 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:09:23.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.210 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139578522999997, 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:23.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:09:23.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139560629000002, 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:23.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:09:23.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:09:23.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139525986999999, 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:23.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:09:23.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139431459000001, 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.225 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139412778000001, 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:23.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:09:23.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:09:23.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139357298999997, 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.229 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:23.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:09:23.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.232 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139322913000001, 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.233 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:23.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:23.236 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139300953999999, 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.237 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:09:23.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:09:23.240 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.304 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.304 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139283871000004, 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:09:23.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.308 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139267410999999, 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:23.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:09:23.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:09:23.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139193495999997, 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.312 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:23.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:09:23.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:09:23.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 47.139140973000003, 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:09:23.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:09:23.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.319 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:23.321 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:23.321 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:23.329+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:23.331 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.7 to 1 (10000) 2024-04-24T18:09:23.344 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-24T18:09:23.422 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:23.430+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:23.480 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:23.480 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:23.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:23.535 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:23.543+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:23.621 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:23.621 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:23.621 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:23.621 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:23.621 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:23.623 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:23.631+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:23.631 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:23.653 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:23.725 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:23.732+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341644221999999, 2024-04-24T18:09:23.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.808 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:23.810 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341626017999999, 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.812 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341604603, 2024-04-24T18:09:23.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.816 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:09:23.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341587451999999, 2024-04-24T18:09:23.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.822 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341571275, 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:23.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.826 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341553437999998, 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:23.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:09:23.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341522834999999, 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:23.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:09:23.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341502675999998, 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.838 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341486398999997, 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.842 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341468808000002, 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341452939, 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341434393, 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:09:23.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341418349000001, 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:23.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:09:23.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:09:23.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341401056999999, 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341379400999998, 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341360571000003, 2024-04-24T18:09:23.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341344235999998, 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.873 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341328410999999, 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.874 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:09:23.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.877 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341309987000002, 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 57.341293612000001, 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:23.881 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:23.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:09:23.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:09:23.884 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.885 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:23.885 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:23.885 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:23.886 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:23.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:23.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:23.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:23.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:23.887 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:23.894+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:23.892 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:23.911 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:23.930 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:23.950 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:23.951 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:23.988 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:23.996+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:24.022 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:24.022 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:24.032 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:24.088 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:24.096+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:24.160 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:24.160 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:24.170 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:24.212 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:24.221+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:24.289 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:24.298+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:24.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:09:24.336 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.1 to 1 (10000) 2024-04-24T18:09:24.354 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:24.354 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:24.355 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-24T18:09:24.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:09:24.366 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:24.409 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:24.416+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:24.491 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:24.500+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:24.563 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:24.563 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:24.574 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:24.592 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:24.601+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:24.693 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:24.702+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:24.705 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:24.716 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:24.778 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:24.780 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:24.791 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-24T18:09:24.333187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":952813043,"num_objects":3186,"num_object_clones":519,"num_object_copies":6372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":2221,"num_read":282150,"num_read_kb":852494,"num_write":27610,"num_write_kb":533704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":39,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23549,"ondisk_log_size":23549,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2478932,"kb_used_data":66000,"kb_used_omap":43,"kb_used_meta":2412756,"kb_avail":752495788,"statfs":{"total":773094113280,"available":770555686912,"internally_reserved":0,"allocated":67584000,"data_stored":567693417,"data_compressed":11378525,"data_compressed_allocated":36290560,"data_compressed_original":545139358,"omap_allocated":44665,"internal_metadata":2470662535},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-142238029,"num_objects":-218,"num_object_clones":-66,"num_object_copies":-436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-218,"num_whiteouts":-61,"num_read":1247,"num_read_kb":1067,"num_write":262,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010124"},"pg_stats":[{"pgid":"4.1f","version":"122'603","reported_seq":6440,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.355188+0000","last_change":"2024-04-24T18:09:21.355188+0000","last_active":"2024-04-24T18:09:21.355188+0000","last_peered":"2024-04-24T18:09:21.355188+0000","last_clean":"2024-04-24T18:09:21.355188+0000","last_became_active":"2024-04-24T18:08:56.900267+0000","last_became_peered":"2024-04-24T18:08:56.900267+0000","last_unstale":"2024-04-24T18:09:21.355188+0000","last_undegraded":"2024-04-24T18:09:21.355188+0000","last_fullsized":"2024-04-24T18:09:21.355188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:07.953531+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.010620701,"stat_sum":{"num_bytes":38699180,"num_objects":118,"num_object_clones":17,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":87,"num_read":7030,"num_read_kb":29081,"num_write":579,"num_write_kb":11953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"4.1e","version":"122'1217","reported_seq":6821,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305100+0000","last_change":"2024-04-24T18:09:22.305524+0000","last_active":"2024-04-24T18:09:22.305100+0000","last_peered":"2024-04-24T18:09:22.305100+0000","last_clean":"2024-04-24T18:09:22.305100+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:09:22.305100+0000","last_undegraded":"2024-04-24T18:09:22.305100+0000","last_fullsized":"2024-04-24T18:09:22.305100+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062748700000000003,"stat_sum":{"num_bytes":28548908,"num_objects":126,"num_object_clones":20,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":88,"num_read":9994,"num_read_kb":25833,"num_write":1803,"num_write_kb":10806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1d","version":"122'644","reported_seq":5643,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312634+0000","last_change":"2024-04-24T18:09:22.312844+0000","last_active":"2024-04-24T18:09:22.312634+0000","last_peered":"2024-04-24T18:09:22.312634+0000","last_clean":"2024-04-24T18:09:22.312634+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:09:22.312634+0000","last_undegraded":"2024-04-24T18:09:22.312634+0000","last_fullsized":"2024-04-24T18:09:22.312634+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002572567,"stat_sum":{"num_bytes":15742650,"num_objects":109,"num_object_clones":16,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":80,"num_read":6089,"num_read_kb":17864,"num_write":594,"num_write_kb":12150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1c","version":"123'948","reported_seq":9643,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.349907+0000","last_change":"2024-04-24T18:09:22.321961+0000","last_active":"2024-04-24T18:09:22.349907+0000","last_peered":"2024-04-24T18:09:22.349907+0000","last_clean":"2024-04-24T18:09:22.349907+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:09:22.349907+0000","last_undegraded":"2024-04-24T18:09:22.349907+0000","last_fullsized":"2024-04-24T18:09:22.349907+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":948,"log_dups_size":0,"ondisk_log_size":948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0025301740000000001,"stat_sum":{"num_bytes":26386468,"num_objects":90,"num_object_clones":16,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":59,"num_read":11691,"num_read_kb":23152,"num_write":1264,"num_write_kb":9983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1b","version":"123'473","reported_seq":11371,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.335548+0000","last_change":"2024-04-24T18:09:22.335734+0000","last_active":"2024-04-24T18:09:22.335548+0000","last_peered":"2024-04-24T18:09:22.335548+0000","last_clean":"2024-04-24T18:09:22.335548+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:09:22.335548+0000","last_undegraded":"2024-04-24T18:09:22.335548+0000","last_fullsized":"2024-04-24T18:09:22.335548+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0040994359999999997,"stat_sum":{"num_bytes":41937960,"num_objects":100,"num_object_clones":20,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":58,"num_read":10912,"num_read_kb":37133,"num_write":461,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1a","version":"122'806","reported_seq":11653,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.352790+0000","last_change":"2024-04-24T18:09:22.332206+0000","last_active":"2024-04-24T18:09:22.352790+0000","last_peered":"2024-04-24T18:09:22.352790+0000","last_clean":"2024-04-24T18:09:22.352790+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:09:22.352790+0000","last_undegraded":"2024-04-24T18:09:22.352790+0000","last_fullsized":"2024-04-24T18:09:22.352790+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000884274,"stat_sum":{"num_bytes":25846410,"num_objects":97,"num_object_clones":20,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":63,"num_read":13672,"num_read_kb":29956,"num_write":956,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.19","version":"123'638","reported_seq":9346,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.319041+0000","last_change":"2024-04-24T18:09:22.319357+0000","last_active":"2024-04-24T18:09:22.319041+0000","last_peered":"2024-04-24T18:09:22.319041+0000","last_clean":"2024-04-24T18:09:22.319041+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:09:22.319041+0000","last_undegraded":"2024-04-24T18:09:22.319041+0000","last_fullsized":"2024-04-24T18:09:22.319041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0085812530000000005,"stat_sum":{"num_bytes":24798587,"num_objects":105,"num_object_clones":14,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":74,"num_read":9982,"num_read_kb":31017,"num_write":751,"num_write_kb":15844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.18","version":"122'551","reported_seq":11264,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.331552+0000","last_change":"2024-04-24T18:09:22.331861+0000","last_active":"2024-04-24T18:09:22.331552+0000","last_peered":"2024-04-24T18:09:22.331552+0000","last_clean":"2024-04-24T18:09:22.331552+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:09:22.331552+0000","last_undegraded":"2024-04-24T18:09:22.331552+0000","last_fullsized":"2024-04-24T18:09:22.331552+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056312699999999996,"stat_sum":{"num_bytes":33494272,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":58,"num_read":12025,"num_read_kb":36026,"num_write":559,"num_write_kb":30739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.17","version":"122'1310","reported_seq":6703,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312902+0000","last_change":"2024-04-24T18:09:22.313208+0000","last_active":"2024-04-24T18:09:22.312902+0000","last_peered":"2024-04-24T18:09:22.312902+0000","last_clean":"2024-04-24T18:09:22.312902+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:09:22.312902+0000","last_undegraded":"2024-04-24T18:09:22.312902+0000","last_fullsized":"2024-04-24T18:09:22.312902+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1310,"log_dups_size":0,"ondisk_log_size":1310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026582979999999999,"stat_sum":{"num_bytes":29953199,"num_objects":99,"num_object_clones":22,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":69,"num_read":10550,"num_read_kb":29370,"num_write":2063,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.2","version":"122'1122","reported_seq":4465,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305217+0000","last_change":"2024-04-24T18:09:22.305579+0000","last_active":"2024-04-24T18:09:22.305217+0000","last_peered":"2024-04-24T18:09:22.305217+0000","last_clean":"2024-04-24T18:09:22.305217+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:09:22.305217+0000","last_undegraded":"2024-04-24T18:09:22.305217+0000","last_fullsized":"2024-04-24T18:09:22.305217+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055743800000000005,"stat_sum":{"num_bytes":28431109,"num_objects":108,"num_object_clones":16,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":72,"num_read":8096,"num_read_kb":23716,"num_write":1628,"num_write_kb":17580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.324566+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:09:20.324566+0000","last_peered":"2024-04-24T18:09:20.324566+0000","last_clean":"2024-04-24T18:09:20.324566+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:09:20.324566+0000","last_undegraded":"2024-04-24T18:09:20.324566+0000","last_fullsized":"2024-04-24T18:09:20.324566+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.324540+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:09:20.324540+0000","last_peered":"2024-04-24T18:09:20.324540+0000","last_clean":"2024-04-24T18:09:20.324540+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:09:20.324540+0000","last_undegraded":"2024-04-24T18:09:20.324540+0000","last_fullsized":"2024-04-24T18:09:20.324540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"123'738","reported_seq":6191,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.369101+0000","last_change":"2024-04-24T18:09:22.306358+0000","last_active":"2024-04-24T18:09:22.369101+0000","last_peered":"2024-04-24T18:09:22.369101+0000","last_clean":"2024-04-24T18:09:22.369101+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:09:22.369101+0000","last_undegraded":"2024-04-24T18:09:22.369101+0000","last_fullsized":"2024-04-24T18:09:22.369101+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000577652,"stat_sum":{"num_bytes":54301688,"num_objects":108,"num_object_clones":21,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":72,"num_read":7372,"num_read_kb":32020,"num_write":757,"num_write_kb":18726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.3","version":"0'0","reported_seq":112,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.333425+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:09:21.333425+0000","last_peered":"2024-04-24T18:09:21.333425+0000","last_clean":"2024-04-24T18:09:21.333425+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:09:21.333425+0000","last_undegraded":"2024-04-24T18:09:21.333425+0000","last_fullsized":"2024-04-24T18:09:21.333425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.332672+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:09:21.332672+0000","last_peered":"2024-04-24T18:09:21.332672+0000","last_clean":"2024-04-24T18:09:21.332672+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:09:21.332672+0000","last_undegraded":"2024-04-24T18:09:21.332672+0000","last_fullsized":"2024-04-24T18:09:21.332672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"122'948","reported_seq":6255,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.306785+0000","last_change":"2024-04-24T18:09:22.306977+0000","last_active":"2024-04-24T18:09:22.306785+0000","last_peered":"2024-04-24T18:09:22.306785+0000","last_clean":"2024-04-24T18:09:22.306785+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:09:22.306785+0000","last_undegraded":"2024-04-24T18:09:22.306785+0000","last_fullsized":"2024-04-24T18:09:22.306785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":948,"log_dups_size":0,"ondisk_log_size":948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000946006,"stat_sum":{"num_bytes":14035202,"num_objects":101,"num_object_clones":15,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":79,"num_read":8490,"num_read_kb":13999,"num_write":1204,"num_write_kb":16363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.2","version":"106'3","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.333335+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:09:21.333335+0000","last_peered":"2024-04-24T18:09:21.333335+0000","last_clean":"2024-04-24T18:09:21.333335+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:09:21.333335+0000","last_undegraded":"2024-04-24T18:09:21.333335+0000","last_fullsized":"2024-04-24T18:09:21.333335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.429586+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:09:20.429586+0000","last_peered":"2024-04-24T18:09:20.429586+0000","last_clean":"2024-04-24T18:09:20.429586+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:09:20.429586+0000","last_undegraded":"2024-04-24T18:09:20.429586+0000","last_fullsized":"2024-04-24T18:09:20.429586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"122'874","reported_seq":9776,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.344876+0000","last_change":"2024-04-24T18:09:21.344876+0000","last_active":"2024-04-24T18:09:21.344876+0000","last_peered":"2024-04-24T18:09:21.344876+0000","last_clean":"2024-04-24T18:09:21.344876+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:09:21.344876+0000","last_undegraded":"2024-04-24T18:09:21.344876+0000","last_fullsized":"2024-04-24T18:09:21.344876+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049185899999999998,"stat_sum":{"num_bytes":35181679,"num_objects":121,"num_object_clones":16,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":83,"num_read":11635,"num_read_kb":43632,"num_write":1032,"num_write_kb":32707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.0","version":"0'0","reported_seq":86,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.327548+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:09:21.327548+0000","last_peered":"2024-04-24T18:09:21.327548+0000","last_clean":"2024-04-24T18:09:21.327548+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:09:21.327548+0000","last_undegraded":"2024-04-24T18:09:21.327548+0000","last_fullsized":"2024-04-24T18:09:21.327548+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.429369+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:09:20.429369+0000","last_peered":"2024-04-24T18:09:20.429369+0000","last_clean":"2024-04-24T18:09:20.429369+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:09:20.429369+0000","last_undegraded":"2024-04-24T18:09:20.429369+0000","last_fullsized":"2024-04-24T18:09:20.429369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"122'486","reported_seq":4958,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.306309+0000","last_change":"2024-04-24T18:09:22.306684+0000","last_active":"2024-04-24T18:09:22.306309+0000","last_peered":"2024-04-24T18:09:22.306309+0000","last_clean":"2024-04-24T18:09:22.306309+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:09:22.306309+0000","last_undegraded":"2024-04-24T18:09:22.306309+0000","last_fullsized":"2024-04-24T18:09:22.306309+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090522900000000002,"stat_sum":{"num_bytes":25148612,"num_objects":96,"num_object_clones":21,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":61,"num_read":6005,"num_read_kb":29263,"num_write":601,"num_write_kb":15062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.1","version":"113'2","reported_seq":114,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.327427+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:09:21.327427+0000","last_peered":"2024-04-24T18:09:21.327427+0000","last_clean":"2024-04-24T18:09:21.327427+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:09:21.327427+0000","last_undegraded":"2024-04-24T18:09:21.327427+0000","last_fullsized":"2024-04-24T18:09:21.327427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":311,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.429932+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:09:20.429932+0000","last_peered":"2024-04-24T18:09:20.429932+0000","last_clean":"2024-04-24T18:09:20.429932+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:09:20.429932+0000","last_undegraded":"2024-04-24T18:09:20.429932+0000","last_fullsized":"2024-04-24T18:09:20.429932+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":101,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.327631+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:09:21.327631+0000","last_peered":"2024-04-24T18:09:21.327631+0000","last_clean":"2024-04-24T18:09:21.327631+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:09:21.327631+0000","last_undegraded":"2024-04-24T18:09:21.327631+0000","last_fullsized":"2024-04-24T18:09:21.327631+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"122'917","reported_seq":9409,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305791+0000","last_change":"2024-04-24T18:09:22.305979+0000","last_active":"2024-04-24T18:09:22.305791+0000","last_peered":"2024-04-24T18:09:22.305791+0000","last_clean":"2024-04-24T18:09:22.305791+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:09:22.305791+0000","last_undegraded":"2024-04-24T18:09:22.305791+0000","last_fullsized":"2024-04-24T18:09:22.305791+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054895200000000001,"stat_sum":{"num_bytes":25398324,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":56,"num_read":12231,"num_read_kb":32090,"num_write":1122,"num_write_kb":22135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1","version":"122'608","reported_seq":5792,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.332261+0000","last_change":"2024-04-24T18:09:22.332511+0000","last_active":"2024-04-24T18:09:22.332261+0000","last_peered":"2024-04-24T18:09:22.332261+0000","last_clean":"2024-04-24T18:09:22.332261+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:09:22.332261+0000","last_undegraded":"2024-04-24T18:09:22.332261+0000","last_fullsized":"2024-04-24T18:09:22.332261+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011516339999999999,"stat_sum":{"num_bytes":32442525,"num_objects":100,"num_object_clones":12,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":81,"num_read":6692,"num_read_kb":29728,"num_write":527,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.332958+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:09:21.332958+0000","last_peered":"2024-04-24T18:09:21.332958+0000","last_clean":"2024-04-24T18:09:21.332958+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:09:21.332958+0000","last_undegraded":"2024-04-24T18:09:21.332958+0000","last_fullsized":"2024-04-24T18:09:21.332958+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"122'769","reported_seq":5873,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312177+0000","last_change":"2024-04-24T18:09:22.312616+0000","last_active":"2024-04-24T18:09:22.312177+0000","last_peered":"2024-04-24T18:09:22.312177+0000","last_clean":"2024-04-24T18:09:22.312177+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:09:22.312177+0000","last_undegraded":"2024-04-24T18:09:22.312177+0000","last_fullsized":"2024-04-24T18:09:22.312177+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024207460000000001,"stat_sum":{"num_bytes":35060659,"num_objects":92,"num_object_clones":18,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":61,"num_read":7361,"num_read_kb":18341,"num_write":826,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.334936+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:09:21.334936+0000","last_peered":"2024-04-24T18:09:21.334936+0000","last_clean":"2024-04-24T18:09:21.334936+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:09:21.334936+0000","last_undegraded":"2024-04-24T18:09:21.334936+0000","last_fullsized":"2024-04-24T18:09:21.334936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"122'579","reported_seq":11624,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305471+0000","last_change":"2024-04-24T18:09:22.305684+0000","last_active":"2024-04-24T18:09:22.305471+0000","last_peered":"2024-04-24T18:09:22.305471+0000","last_clean":"2024-04-24T18:09:22.305471+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:09:22.305471+0000","last_undegraded":"2024-04-24T18:09:22.305471+0000","last_fullsized":"2024-04-24T18:09:22.305471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042369599999999999,"stat_sum":{"num_bytes":33105183,"num_objects":109,"num_object_clones":18,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":74,"num_read":12129,"num_read_kb":39327,"num_write":580,"num_write_kb":18752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.9","version":"123'602","reported_seq":6007,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.333378+0000","last_change":"2024-04-24T18:09:22.333585+0000","last_active":"2024-04-24T18:09:22.333378+0000","last_peered":"2024-04-24T18:09:22.333378+0000","last_clean":"2024-04-24T18:09:22.333378+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:09:22.333378+0000","last_undegraded":"2024-04-24T18:09:22.333378+0000","last_fullsized":"2024-04-24T18:09:22.333378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0022942420000000002,"stat_sum":{"num_bytes":34481159,"num_objects":99,"num_object_clones":13,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":68,"num_read":6059,"num_read_kb":18027,"num_write":633,"num_write_kb":12791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.a","version":"122'639","reported_seq":3642,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.450214+0000","last_change":"2024-04-24T18:09:21.345209+0000","last_active":"2024-04-24T18:09:21.450214+0000","last_peered":"2024-04-24T18:09:21.450214+0000","last_clean":"2024-04-24T18:09:21.450214+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:09:21.450214+0000","last_undegraded":"2024-04-24T18:09:21.450214+0000","last_fullsized":"2024-04-24T18:09:21.450214+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064287600000000002,"stat_sum":{"num_bytes":14747015,"num_objects":75,"num_object_clones":5,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":58,"num_read":3732,"num_read_kb":11537,"num_write":717,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"4.b","version":"123'603","reported_seq":5286,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.379488+0000","last_change":"2024-04-24T18:09:22.306259+0000","last_active":"2024-04-24T18:09:22.379488+0000","last_peered":"2024-04-24T18:09:22.379488+0000","last_clean":"2024-04-24T18:09:22.379488+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:09:22.379488+0000","last_undegraded":"2024-04-24T18:09:22.379488+0000","last_fullsized":"2024-04-24T18:09:22.379488+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820442,"stat_sum":{"num_bytes":22043736,"num_objects":82,"num_object_clones":11,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":59,"num_read":7726,"num_read_kb":16517,"num_write":637,"num_write_kb":13354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.c","version":"122'515","reported_seq":6672,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.332562+0000","last_change":"2024-04-24T18:09:22.332811+0000","last_active":"2024-04-24T18:09:22.332562+0000","last_peered":"2024-04-24T18:09:22.332562+0000","last_clean":"2024-04-24T18:09:22.332562+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:09:22.332562+0000","last_undegraded":"2024-04-24T18:09:22.332562+0000","last_fullsized":"2024-04-24T18:09:22.332562+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361592,"stat_sum":{"num_bytes":24665650,"num_objects":81,"num_object_clones":11,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":59,"num_read":7293,"num_read_kb":28455,"num_write":543,"num_write_kb":28659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.d","version":"123'559","reported_seq":8917,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.333789+0000","last_change":"2024-04-24T18:09:22.333789+0000","last_active":"2024-04-24T18:09:22.333789+0000","last_peered":"2024-04-24T18:09:22.333789+0000","last_clean":"2024-04-24T18:09:22.333789+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:09:22.333789+0000","last_undegraded":"2024-04-24T18:09:22.333789+0000","last_fullsized":"2024-04-24T18:09:22.333789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020913059999999998,"stat_sum":{"num_bytes":32694003,"num_objects":115,"num_object_clones":20,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":82,"num_read":8793,"num_read_kb":23795,"num_write":519,"num_write_kb":9562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.e","version":"122'518","reported_seq":6797,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.333859+0000","last_change":"2024-04-24T18:09:22.333985+0000","last_active":"2024-04-24T18:09:22.333859+0000","last_peered":"2024-04-24T18:09:22.333859+0000","last_clean":"2024-04-24T18:09:22.333859+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:09:22.333859+0000","last_undegraded":"2024-04-24T18:09:22.333859+0000","last_fullsized":"2024-04-24T18:09:22.333859+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020446930000000002,"stat_sum":{"num_bytes":26474926,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":58,"num_read":6601,"num_read_kb":16217,"num_write":532,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.f","version":"122'777","reported_seq":5947,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305971+0000","last_change":"2024-04-24T18:09:22.306222+0000","last_active":"2024-04-24T18:09:22.305971+0000","last_peered":"2024-04-24T18:09:22.305971+0000","last_clean":"2024-04-24T18:09:22.305971+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:09:22.305971+0000","last_undegraded":"2024-04-24T18:09:22.305971+0000","last_fullsized":"2024-04-24T18:09:22.305971+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044597899999999999,"stat_sum":{"num_bytes":20023412,"num_objects":92,"num_object_clones":10,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":65,"num_read":7901,"num_read_kb":20503,"num_write":860,"num_write_kb":8614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.10","version":"122'592","reported_seq":7085,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.310293+0000","last_change":"2024-04-24T18:09:22.310654+0000","last_active":"2024-04-24T18:09:22.310293+0000","last_peered":"2024-04-24T18:09:22.310293+0000","last_clean":"2024-04-24T18:09:22.310293+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:09:22.310293+0000","last_undegraded":"2024-04-24T18:09:22.310293+0000","last_fullsized":"2024-04-24T18:09:22.310293+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061278299999999999,"stat_sum":{"num_bytes":19642892,"num_objects":105,"num_object_clones":20,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":76,"num_read":7841,"num_read_kb":18286,"num_write":636,"num_write_kb":14132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.11","version":"122'675","reported_seq":8818,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312106+0000","last_change":"2024-04-24T18:09:22.312537+0000","last_active":"2024-04-24T18:09:22.312106+0000","last_peered":"2024-04-24T18:09:22.312106+0000","last_clean":"2024-04-24T18:09:22.312106+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:09:22.312106+0000","last_undegraded":"2024-04-24T18:09:22.312106+0000","last_fullsized":"2024-04-24T18:09:22.312106+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024990139999999999,"stat_sum":{"num_bytes":80803197,"num_objects":116,"num_object_clones":22,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":80,"num_read":9362,"num_read_kb":48622,"num_write":822,"num_write_kb":30856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.12","version":"122'572","reported_seq":5976,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305744+0000","last_change":"2024-04-24T18:09:22.306450+0000","last_active":"2024-04-24T18:09:22.305744+0000","last_peered":"2024-04-24T18:09:22.305744+0000","last_clean":"2024-04-24T18:09:22.305744+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:09:22.305744+0000","last_undegraded":"2024-04-24T18:09:22.305744+0000","last_fullsized":"2024-04-24T18:09:22.305744+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012581599999999999,"stat_sum":{"num_bytes":24440399,"num_objects":115,"num_object_clones":15,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":82,"num_read":9712,"num_read_kb":31145,"num_write":536,"num_write_kb":8699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.13","version":"123'920","reported_seq":8884,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.335782+0000","last_change":"2024-04-24T18:09:22.335959+0000","last_active":"2024-04-24T18:09:22.335782+0000","last_peered":"2024-04-24T18:09:22.335782+0000","last_clean":"2024-04-24T18:09:22.335782+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:09:22.335782+0000","last_undegraded":"2024-04-24T18:09:22.335782+0000","last_fullsized":"2024-04-24T18:09:22.335782+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0038216650000000001,"stat_sum":{"num_bytes":19146518,"num_objects":91,"num_object_clones":19,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":63,"num_read":10390,"num_read_kb":22682,"num_write":1282,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.14","version":"123'578","reported_seq":7476,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.381889+0000","last_change":"2024-04-24T18:09:22.312966+0000","last_active":"2024-04-24T18:09:22.381889+0000","last_peered":"2024-04-24T18:09:22.381889+0000","last_clean":"2024-04-24T18:09:22.381889+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:09:22.381889+0000","last_undegraded":"2024-04-24T18:09:22.381889+0000","last_fullsized":"2024-04-24T18:09:22.381889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026239990000000001,"stat_sum":{"num_bytes":36034936,"num_objects":80,"num_object_clones":15,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":53,"num_read":9299,"num_read_kb":25220,"num_write":507,"num_write_kb":9834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.15","version":"122'566","reported_seq":5632,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305511+0000","last_change":"2024-04-24T18:09:22.305683+0000","last_active":"2024-04-24T18:09:22.305511+0000","last_peered":"2024-04-24T18:09:22.305511+0000","last_clean":"2024-04-24T18:09:22.305511+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:09:22.305511+0000","last_undegraded":"2024-04-24T18:09:22.305511+0000","last_fullsized":"2024-04-24T18:09:22.305511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000327015,"stat_sum":{"num_bytes":6370142,"num_objects":84,"num_object_clones":13,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":63,"num_read":7237,"num_read_kb":17222,"num_write":474,"num_write_kb":9146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.16","version":"122'1080","reported_seq":6259,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305736+0000","last_change":"2024-04-24T18:09:22.305932+0000","last_active":"2024-04-24T18:09:22.305736+0000","last_peered":"2024-04-24T18:09:22.305736+0000","last_clean":"2024-04-24T18:09:22.305736+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:09:22.305736+0000","last_undegraded":"2024-04-24T18:09:22.305736+0000","last_fullsized":"2024-04-24T18:09:22.305736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071456099999999995,"stat_sum":{"num_bytes":42141907,"num_objects":109,"num_object_clones":18,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":77,"num_read":8132,"num_read_kb":32618,"num_write":1444,"num_write_kb":16906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":952222507,"num_objects":3179,"num_object_clones":519,"num_object_copies":6358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3179,"num_whiteouts":2218,"num_read":282034,"num_read_kb":852394,"num_write":27449,"num_write_kb":531286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55525376,"data_stored":559769443,"data_compressed":11347625,"data_compressed_allocated":36212736,"data_compressed_original":543378078,"omap_allocated":9738,"internal_metadata":0},"log_size":23427,"ondisk_log_size":23427,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308356,"kb_used_data":6904,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94063484,"statfs":{"total":96636764160,"available":96321007616,"internally_reserved":0,"allocated":7069696,"data_stored":60059913,"data_compressed":945105,"data_compressed_allocated":3694592,"data_compressed_original":57833704,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.47399999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.55100000000000005,"15min":0.503},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.6740000000000004,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.55300000000000005,"15min":0.51100000000000001},"min":{"1min":0.27300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.3879999999999999,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.751,"5min":0.53500000000000003,"15min":0.499},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":7.4749999999999996,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.24099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.311},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.498,"15min":0.502},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.39200000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.35099999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.44600000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893127,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309660,"kb_used_data":6928,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94062180,"statfs":{"total":96636764160,"available":96319672320,"internally_reserved":0,"allocated":7094272,"data_stored":57840122,"data_compressed":1327608,"data_compressed_allocated":3989504,"data_compressed_original":55713461,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308932,"kb_used_data":11568,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062908,"statfs":{"total":96636764160,"available":96320417792,"internally_reserved":0,"allocated":11845632,"data_stored":83219001,"data_compressed":1774201,"data_compressed_allocated":5709824,"data_compressed_original":78704524,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.59999999999999998,"15min":0.64000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.036,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.377,"5min":1.0700000000000001,"15min":1.1859999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.70099999999999996,"5min":14.15,"15min":14.15},"last":0.432}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.499,"15min":0.505},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1799999999999999,"5min":1.306,"15min":1.306},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":1.135,"15min":1.246},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.2010000000000001,"5min":14.118,"15min":14.118},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":1.1000000000000001,"15min":1.208},"min":{"1min":0.34699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.65900000000000003,"5min":14.089,"15min":14.089},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.65700000000000003,"15min":0.69799999999999995},"min":{"1min":0.32500000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60899999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.59899999999999998,"15min":0.61899999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96699999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.622,"5min":1.23,"15min":1.331},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.8240000000000001,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.73599999999999999,"15min":0.77900000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.032,"5min":1.49,"15min":1.49},"last":0.53100000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":1.2270000000000001,"15min":1.345},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84799999999999998,"5min":14.051,"15min":14.051},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.79700000000000004,"15min":0.84099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.141,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":5744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":5881856,"data_stored":51121180,"data_compressed":916961,"data_compressed_allocated":3117056,"data_compressed_original":49313082,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.40999999999999998,"15min":0.42099999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.67000000000000004,"5min":1.375,"15min":1.375},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.375,"5min":1.5389999999999999,"15min":1.7330000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54700000000000004,"5min":26.936,"15min":26.936},"last":0.27200000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":1.0960000000000001,"15min":1.206},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0629999999999999,"5min":11.93,"15min":11.93},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.40899999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":6.4889999999999999,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.95899999999999996,"15min":1.036},"min":{"1min":0.32200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.5880000000000001,"5min":14.371,"15min":14.371},"last":0.371},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.30099999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.6390000000000002,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":1.0580000000000001,"15min":1.1579999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":14.414,"15min":14.414},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.74099999999999999,"5min":1.329,"15min":1.329},"last":0.42199999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":6.5629999999999997,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.0920000000000001,"15min":1.173},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.52,"5min":14.452,"15min":14.452},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.82799999999999996,"15min":0.82299999999999995},"min":{"1min":0.371,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.5300000000000002,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.624,"5min":1.1080000000000001,"15min":1.1879999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.5580000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313420,"kb_used_data":3764,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058420,"statfs":{"total":96636764160,"available":96315822080,"internally_reserved":0,"allocated":3854336,"data_stored":19766273,"data_compressed":452614,"data_compressed_allocated":1462272,"data_compressed_original":18222740,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.443,"15min":0.41999999999999998},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.42199999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.40999999999999998},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.8929999999999998,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.40400000000000003,"15min":0.39200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.33000000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74299999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.52900000000000003,"15min":0.48699999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":4.1399999999999997,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.60999999999999999,"15min":0.57799999999999996},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.627},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":1.155,"15min":1.155},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.372},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.0679999999999996,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.72899999999999998,"15min":0.65900000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":10.516,"5min":10.516,"15min":10.516},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.64000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309988,"kb_used_data":12628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94061852,"statfs":{"total":96636764160,"available":96319336448,"internally_reserved":0,"allocated":12931072,"data_stored":123320821,"data_compressed":2137338,"data_compressed_allocated":7512064,"data_compressed_original":119016401,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.37,"15min":0.35799999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.219},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28399999999999997}]},{"osd":1,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.41699999999999998,"15min":0.42599999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.2829999999999999,"5min":1.792,"15min":1.792},"last":0.254},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.30099999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.377},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"min":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6699999999999999,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.48799999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.63200000000000001,"15min":0.59799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.441},{"interface":"front","average":{"1min":1.3100000000000001,"5min":0.75,"15min":0.65600000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":15.4,"5min":15.4,"15min":15.4},"last":0.46400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315620,"kb_used_data":10068,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94056220,"statfs":{"total":96636764160,"available":96313569280,"internally_reserved":0,"allocated":10309632,"data_stored":101470791,"data_compressed":2325730,"data_compressed_allocated":6209536,"data_compressed_original":98340115,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.441}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.40500000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38200000000000001}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.35899999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.46999999999999997}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309856,"kb_used_data":8396,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94061984,"statfs":{"total":96636764160,"available":96319471616,"internally_reserved":0,"allocated":8597504,"data_stored":70895316,"data_compressed":1498968,"data_compressed_allocated":4595712,"data_compressed_original":67995331,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.64600000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.436,"15min":0.44},"min":{"1min":0.30499999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66600000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.504},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48799999999999999,"15min":0.495},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59499999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.52500000000000002,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.52700000000000002,"5min":1.296,"15min":1.296},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71999999999999997,"5min":1.377,"15min":1.377},"last":0.371}]},{"osd":3,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.68700000000000006,"5min":1.899,"15min":1.899},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.628,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.881,"15min":1.881},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49099999999999999,"15min":0.51000000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52800000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.74299999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.32500000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.54900000000000004,"5min":1.016,"15min":1.016},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81200000000000006,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4419584,"data_stored":50390578,"data_compressed":916961,"data_compressed_allocated":3117056,"data_compressed_original":49313082,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10244096,"data_stored":81598593,"data_compressed":1763503,"data_compressed_allocated":5672960,"data_compressed_original":77918092,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2392064,"data_stored":18981287,"data_compressed":452614,"data_compressed_allocated":1462272,"data_compressed_original":18222740,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11382784,"data_stored":122458937,"data_compressed":2137076,"data_compressed_allocated":7507968,"data_compressed_original":118967249,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8843264,"data_stored":100384544,"data_compressed":2314454,"data_compressed_allocated":6209536,"data_compressed_original":97987859,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7122944,"data_stored":70158044,"data_compressed":1498968,"data_compressed_allocated":4595712,"data_compressed_original":67995331,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5566464,"data_stored":57058434,"data_compressed":1327608,"data_compressed_allocated":3989504,"data_compressed_original":55713461,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5554176,"data_stored":58739026,"data_compressed":936441,"data_compressed_allocated":3657728,"data_compressed_original":57260264,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:24.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:09:24.796 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:24.803+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:24.848 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:24.848 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:24.859 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:24.896 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:24.904+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:24.997 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:25.005+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:25.002 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:25.002 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:25.013 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:25.098 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:25.107+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:25.129 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:25.130 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:09:25.138 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:25.138 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:25.141 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":181,"stamp":"2024-04-24T18:09:24.333187+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":952813043,"num_objects":3186,"num_object_clones":519,"num_object_copies":6372,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3181,"num_whiteouts":2221,"num_read":282150,"num_read_kb":852494,"num_write":27610,"num_write_kb":533704,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":39,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23549,"ondisk_log_size":23549,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2478932,"kb_used_data":66000,"kb_used_omap":43,"kb_used_meta":2412756,"kb_avail":752495788,"statfs":{"total":773094113280,"available":770555686912,"internally_reserved":0,"allocated":67584000,"data_stored":567693417,"data_compressed":11378525,"data_compressed_allocated":36290560,"data_compressed_original":545139358,"omap_allocated":44665,"internal_metadata":2470662535},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-142238029,"num_objects":-218,"num_object_clones":-66,"num_object_copies":-436,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-218,"num_whiteouts":-61,"num_read":1247,"num_read_kb":1067,"num_write":262,"num_write_kb":60,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.010124"},"pg_stats":[{"pgid":"4.1f","version":"122'603","reported_seq":6440,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.355188+0000","last_change":"2024-04-24T18:09:21.355188+0000","last_active":"2024-04-24T18:09:21.355188+0000","last_peered":"2024-04-24T18:09:21.355188+0000","last_clean":"2024-04-24T18:09:21.355188+0000","last_became_active":"2024-04-24T18:08:56.900267+0000","last_became_peered":"2024-04-24T18:08:56.900267+0000","last_unstale":"2024-04-24T18:09:21.355188+0000","last_undegraded":"2024-04-24T18:09:21.355188+0000","last_fullsized":"2024-04-24T18:09:21.355188+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:07.953531+0000","scrub_duration":0,"objects_trimmed":6,"snaptrim_duration":0.010620701,"stat_sum":{"num_bytes":38699180,"num_objects":118,"num_object_clones":17,"num_object_copies":236,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":118,"num_whiteouts":87,"num_read":7030,"num_read_kb":29081,"num_write":579,"num_write_kb":11953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"4.1e","version":"122'1217","reported_seq":6821,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305100+0000","last_change":"2024-04-24T18:09:22.305524+0000","last_active":"2024-04-24T18:09:22.305100+0000","last_peered":"2024-04-24T18:09:22.305100+0000","last_clean":"2024-04-24T18:09:22.305100+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:09:22.305100+0000","last_undegraded":"2024-04-24T18:09:22.305100+0000","last_fullsized":"2024-04-24T18:09:22.305100+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1217,"log_dups_size":0,"ondisk_log_size":1217,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062748700000000003,"stat_sum":{"num_bytes":28548908,"num_objects":126,"num_object_clones":20,"num_object_copies":252,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":126,"num_whiteouts":88,"num_read":9994,"num_read_kb":25833,"num_write":1803,"num_write_kb":10806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1d","version":"122'644","reported_seq":5643,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312634+0000","last_change":"2024-04-24T18:09:22.312844+0000","last_active":"2024-04-24T18:09:22.312634+0000","last_peered":"2024-04-24T18:09:22.312634+0000","last_clean":"2024-04-24T18:09:22.312634+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:09:22.312634+0000","last_undegraded":"2024-04-24T18:09:22.312634+0000","last_fullsized":"2024-04-24T18:09:22.312634+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002572567,"stat_sum":{"num_bytes":15742650,"num_objects":109,"num_object_clones":16,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":80,"num_read":6089,"num_read_kb":17864,"num_write":594,"num_write_kb":12150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1c","version":"123'948","reported_seq":9643,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.349907+0000","last_change":"2024-04-24T18:09:22.321961+0000","last_active":"2024-04-24T18:09:22.349907+0000","last_peered":"2024-04-24T18:09:22.349907+0000","last_clean":"2024-04-24T18:09:22.349907+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:09:22.349907+0000","last_undegraded":"2024-04-24T18:09:22.349907+0000","last_fullsized":"2024-04-24T18:09:22.349907+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":948,"log_dups_size":0,"ondisk_log_size":948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0025301740000000001,"stat_sum":{"num_bytes":26386468,"num_objects":90,"num_object_clones":16,"num_object_copies":180,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":90,"num_whiteouts":59,"num_read":11691,"num_read_kb":23152,"num_write":1264,"num_write_kb":9983,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1b","version":"123'473","reported_seq":11371,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.335548+0000","last_change":"2024-04-24T18:09:22.335734+0000","last_active":"2024-04-24T18:09:22.335548+0000","last_peered":"2024-04-24T18:09:22.335548+0000","last_clean":"2024-04-24T18:09:22.335548+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:09:22.335548+0000","last_undegraded":"2024-04-24T18:09:22.335548+0000","last_fullsized":"2024-04-24T18:09:22.335548+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":473,"log_dups_size":0,"ondisk_log_size":473,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0040994359999999997,"stat_sum":{"num_bytes":41937960,"num_objects":100,"num_object_clones":20,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":58,"num_read":10912,"num_read_kb":37133,"num_write":461,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1a","version":"122'806","reported_seq":11653,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.352790+0000","last_change":"2024-04-24T18:09:22.332206+0000","last_active":"2024-04-24T18:09:22.352790+0000","last_peered":"2024-04-24T18:09:22.352790+0000","last_clean":"2024-04-24T18:09:22.352790+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:09:22.352790+0000","last_undegraded":"2024-04-24T18:09:22.352790+0000","last_fullsized":"2024-04-24T18:09:22.352790+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":806,"log_dups_size":0,"ondisk_log_size":806,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000884274,"stat_sum":{"num_bytes":25846410,"num_objects":97,"num_object_clones":20,"num_object_copies":194,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":97,"num_whiteouts":63,"num_read":13672,"num_read_kb":29956,"num_write":956,"num_write_kb":12165,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.19","version":"123'638","reported_seq":9346,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.319041+0000","last_change":"2024-04-24T18:09:22.319357+0000","last_active":"2024-04-24T18:09:22.319041+0000","last_peered":"2024-04-24T18:09:22.319041+0000","last_clean":"2024-04-24T18:09:22.319041+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:09:22.319041+0000","last_undegraded":"2024-04-24T18:09:22.319041+0000","last_fullsized":"2024-04-24T18:09:22.319041+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":638,"log_dups_size":0,"ondisk_log_size":638,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0085812530000000005,"stat_sum":{"num_bytes":24798587,"num_objects":105,"num_object_clones":14,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":74,"num_read":9982,"num_read_kb":31017,"num_write":751,"num_write_kb":15844,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.18","version":"122'551","reported_seq":11264,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.331552+0000","last_change":"2024-04-24T18:09:22.331861+0000","last_active":"2024-04-24T18:09:22.331552+0000","last_peered":"2024-04-24T18:09:22.331552+0000","last_clean":"2024-04-24T18:09:22.331552+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:09:22.331552+0000","last_undegraded":"2024-04-24T18:09:22.331552+0000","last_fullsized":"2024-04-24T18:09:22.331552+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":551,"log_dups_size":0,"ondisk_log_size":551,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056312699999999996,"stat_sum":{"num_bytes":33494272,"num_objects":86,"num_object_clones":15,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":58,"num_read":12025,"num_read_kb":36026,"num_write":559,"num_write_kb":30739,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.17","version":"122'1310","reported_seq":6703,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312902+0000","last_change":"2024-04-24T18:09:22.313208+0000","last_active":"2024-04-24T18:09:22.312902+0000","last_peered":"2024-04-24T18:09:22.312902+0000","last_clean":"2024-04-24T18:09:22.312902+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:09:22.312902+0000","last_undegraded":"2024-04-24T18:09:22.312902+0000","last_fullsized":"2024-04-24T18:09:22.312902+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1310,"log_dups_size":0,"ondisk_log_size":1310,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026582979999999999,"stat_sum":{"num_bytes":29953199,"num_objects":99,"num_object_clones":22,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":69,"num_read":10550,"num_read_kb":29370,"num_write":2063,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.2","version":"122'1122","reported_seq":4465,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305217+0000","last_change":"2024-04-24T18:09:22.305579+0000","last_active":"2024-04-24T18:09:22.305217+0000","last_peered":"2024-04-24T18:09:22.305217+0000","last_clean":"2024-04-24T18:09:22.305217+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:09:22.305217+0000","last_undegraded":"2024-04-24T18:09:22.305217+0000","last_fullsized":"2024-04-24T18:09:22.305217+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1122,"log_dups_size":0,"ondisk_log_size":1122,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00055743800000000005,"stat_sum":{"num_bytes":28431109,"num_objects":108,"num_object_clones":16,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":72,"num_read":8096,"num_read_kb":23716,"num_write":1628,"num_write_kb":17580,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"1.7","version":"0'0","reported_seq":88,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.324566+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:09:20.324566+0000","last_peered":"2024-04-24T18:09:20.324566+0000","last_clean":"2024-04-24T18:09:20.324566+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:09:20.324566+0000","last_undegraded":"2024-04-24T18:09:20.324566+0000","last_fullsized":"2024-04-24T18:09:20.324566+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":88,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.324540+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:09:20.324540+0000","last_peered":"2024-04-24T18:09:20.324540+0000","last_clean":"2024-04-24T18:09:20.324540+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:09:20.324540+0000","last_undegraded":"2024-04-24T18:09:20.324540+0000","last_fullsized":"2024-04-24T18:09:20.324540+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"123'738","reported_seq":6191,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.369101+0000","last_change":"2024-04-24T18:09:22.306358+0000","last_active":"2024-04-24T18:09:22.369101+0000","last_peered":"2024-04-24T18:09:22.369101+0000","last_clean":"2024-04-24T18:09:22.369101+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:09:22.369101+0000","last_undegraded":"2024-04-24T18:09:22.369101+0000","last_fullsized":"2024-04-24T18:09:22.369101+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":738,"log_dups_size":0,"ondisk_log_size":738,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000577652,"stat_sum":{"num_bytes":54301688,"num_objects":108,"num_object_clones":21,"num_object_copies":216,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":108,"num_whiteouts":72,"num_read":7372,"num_read_kb":32020,"num_write":757,"num_write_kb":18726,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.3","version":"0'0","reported_seq":112,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.333425+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:09:21.333425+0000","last_peered":"2024-04-24T18:09:21.333425+0000","last_clean":"2024-04-24T18:09:21.333425+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:09:21.333425+0000","last_undegraded":"2024-04-24T18:09:21.333425+0000","last_fullsized":"2024-04-24T18:09:21.333425+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.332672+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:09:21.332672+0000","last_peered":"2024-04-24T18:09:21.332672+0000","last_clean":"2024-04-24T18:09:21.332672+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:09:21.332672+0000","last_undegraded":"2024-04-24T18:09:21.332672+0000","last_fullsized":"2024-04-24T18:09:21.332672+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"122'948","reported_seq":6255,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.306785+0000","last_change":"2024-04-24T18:09:22.306977+0000","last_active":"2024-04-24T18:09:22.306785+0000","last_peered":"2024-04-24T18:09:22.306785+0000","last_clean":"2024-04-24T18:09:22.306785+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:09:22.306785+0000","last_undegraded":"2024-04-24T18:09:22.306785+0000","last_fullsized":"2024-04-24T18:09:22.306785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":948,"log_dups_size":0,"ondisk_log_size":948,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000946006,"stat_sum":{"num_bytes":14035202,"num_objects":101,"num_object_clones":15,"num_object_copies":202,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":101,"num_whiteouts":79,"num_read":8490,"num_read_kb":13999,"num_write":1204,"num_write_kb":16363,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.2","version":"106'3","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.333335+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:09:21.333335+0000","last_peered":"2024-04-24T18:09:21.333335+0000","last_clean":"2024-04-24T18:09:21.333335+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:09:21.333335+0000","last_undegraded":"2024-04-24T18:09:21.333335+0000","last_fullsized":"2024-04-24T18:09:21.333335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":113,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.429586+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:09:20.429586+0000","last_peered":"2024-04-24T18:09:20.429586+0000","last_clean":"2024-04-24T18:09:20.429586+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:09:20.429586+0000","last_undegraded":"2024-04-24T18:09:20.429586+0000","last_fullsized":"2024-04-24T18:09:20.429586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"122'874","reported_seq":9776,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.344876+0000","last_change":"2024-04-24T18:09:21.344876+0000","last_active":"2024-04-24T18:09:21.344876+0000","last_peered":"2024-04-24T18:09:21.344876+0000","last_clean":"2024-04-24T18:09:21.344876+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:09:21.344876+0000","last_undegraded":"2024-04-24T18:09:21.344876+0000","last_fullsized":"2024-04-24T18:09:21.344876+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":874,"log_dups_size":0,"ondisk_log_size":874,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049185899999999998,"stat_sum":{"num_bytes":35181679,"num_objects":121,"num_object_clones":16,"num_object_copies":242,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":121,"num_whiteouts":83,"num_read":11635,"num_read_kb":43632,"num_write":1032,"num_write_kb":32707,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"3.0","version":"0'0","reported_seq":86,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.327548+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:09:21.327548+0000","last_peered":"2024-04-24T18:09:21.327548+0000","last_clean":"2024-04-24T18:09:21.327548+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:09:21.327548+0000","last_undegraded":"2024-04-24T18:09:21.327548+0000","last_fullsized":"2024-04-24T18:09:21.327548+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":113,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.429369+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:09:20.429369+0000","last_peered":"2024-04-24T18:09:20.429369+0000","last_clean":"2024-04-24T18:09:20.429369+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:09:20.429369+0000","last_undegraded":"2024-04-24T18:09:20.429369+0000","last_fullsized":"2024-04-24T18:09:20.429369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"122'486","reported_seq":4958,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.306309+0000","last_change":"2024-04-24T18:09:22.306684+0000","last_active":"2024-04-24T18:09:22.306309+0000","last_peered":"2024-04-24T18:09:22.306309+0000","last_clean":"2024-04-24T18:09:22.306309+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:09:22.306309+0000","last_undegraded":"2024-04-24T18:09:22.306309+0000","last_fullsized":"2024-04-24T18:09:22.306309+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":486,"log_dups_size":0,"ondisk_log_size":486,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00090522900000000002,"stat_sum":{"num_bytes":25148612,"num_objects":96,"num_object_clones":21,"num_object_copies":192,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":96,"num_whiteouts":61,"num_read":6005,"num_read_kb":29263,"num_write":601,"num_write_kb":15062,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"3.1","version":"113'2","reported_seq":114,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.327427+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:09:21.327427+0000","last_peered":"2024-04-24T18:09:21.327427+0000","last_clean":"2024-04-24T18:09:21.327427+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:09:21.327427+0000","last_undegraded":"2024-04-24T18:09:21.327427+0000","last_fullsized":"2024-04-24T18:09:21.327427+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":311,"reported_epoch":121,"state":"active+clean","last_fresh":"2024-04-24T18:09:20.429932+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:09:20.429932+0000","last_peered":"2024-04-24T18:09:20.429932+0000","last_clean":"2024-04-24T18:09:20.429932+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:09:20.429932+0000","last_undegraded":"2024-04-24T18:09:20.429932+0000","last_fullsized":"2024-04-24T18:09:20.429932+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":101,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.327631+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:09:21.327631+0000","last_peered":"2024-04-24T18:09:21.327631+0000","last_clean":"2024-04-24T18:09:21.327631+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:09:21.327631+0000","last_undegraded":"2024-04-24T18:09:21.327631+0000","last_fullsized":"2024-04-24T18:09:21.327631+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"122'917","reported_seq":9409,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305791+0000","last_change":"2024-04-24T18:09:22.305979+0000","last_active":"2024-04-24T18:09:22.305791+0000","last_peered":"2024-04-24T18:09:22.305791+0000","last_clean":"2024-04-24T18:09:22.305791+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:09:22.305791+0000","last_undegraded":"2024-04-24T18:09:22.305791+0000","last_fullsized":"2024-04-24T18:09:22.305791+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":917,"log_dups_size":0,"ondisk_log_size":917,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00054895200000000001,"stat_sum":{"num_bytes":25398324,"num_objects":83,"num_object_clones":13,"num_object_copies":166,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":83,"num_whiteouts":56,"num_read":12231,"num_read_kb":32090,"num_write":1122,"num_write_kb":22135,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.1","version":"122'608","reported_seq":5792,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.332261+0000","last_change":"2024-04-24T18:09:22.332511+0000","last_active":"2024-04-24T18:09:22.332261+0000","last_peered":"2024-04-24T18:09:22.332261+0000","last_clean":"2024-04-24T18:09:22.332261+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:09:22.332261+0000","last_undegraded":"2024-04-24T18:09:22.332261+0000","last_fullsized":"2024-04-24T18:09:22.332261+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":608,"log_dups_size":0,"ondisk_log_size":608,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011516339999999999,"stat_sum":{"num_bytes":32442525,"num_objects":100,"num_object_clones":12,"num_object_copies":200,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":81,"num_read":6692,"num_read_kb":29728,"num_write":527,"num_write_kb":23512,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"1.4","version":"0'0","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.332958+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:09:21.332958+0000","last_peered":"2024-04-24T18:09:21.332958+0000","last_clean":"2024-04-24T18:09:21.332958+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:09:21.332958+0000","last_undegraded":"2024-04-24T18:09:21.332958+0000","last_fullsized":"2024-04-24T18:09:21.332958+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"122'769","reported_seq":5873,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312177+0000","last_change":"2024-04-24T18:09:22.312616+0000","last_active":"2024-04-24T18:09:22.312177+0000","last_peered":"2024-04-24T18:09:22.312177+0000","last_clean":"2024-04-24T18:09:22.312177+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:09:22.312177+0000","last_undegraded":"2024-04-24T18:09:22.312177+0000","last_fullsized":"2024-04-24T18:09:22.312177+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":769,"log_dups_size":0,"ondisk_log_size":769,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024207460000000001,"stat_sum":{"num_bytes":35060659,"num_objects":92,"num_object_clones":18,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":61,"num_read":7361,"num_read_kb":18341,"num_write":826,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"1.5","version":"0'0","reported_seq":115,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.334936+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:09:21.334936+0000","last_peered":"2024-04-24T18:09:21.334936+0000","last_clean":"2024-04-24T18:09:21.334936+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:09:21.334936+0000","last_undegraded":"2024-04-24T18:09:21.334936+0000","last_fullsized":"2024-04-24T18:09:21.334936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"122'579","reported_seq":11624,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305471+0000","last_change":"2024-04-24T18:09:22.305684+0000","last_active":"2024-04-24T18:09:22.305471+0000","last_peered":"2024-04-24T18:09:22.305471+0000","last_clean":"2024-04-24T18:09:22.305471+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:09:22.305471+0000","last_undegraded":"2024-04-24T18:09:22.305471+0000","last_fullsized":"2024-04-24T18:09:22.305471+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":579,"log_dups_size":0,"ondisk_log_size":579,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00042369599999999999,"stat_sum":{"num_bytes":33105183,"num_objects":109,"num_object_clones":18,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":74,"num_read":12129,"num_read_kb":39327,"num_write":580,"num_write_kb":18752,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.9","version":"123'602","reported_seq":6007,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.333378+0000","last_change":"2024-04-24T18:09:22.333585+0000","last_active":"2024-04-24T18:09:22.333378+0000","last_peered":"2024-04-24T18:09:22.333378+0000","last_clean":"2024-04-24T18:09:22.333378+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:09:22.333378+0000","last_undegraded":"2024-04-24T18:09:22.333378+0000","last_fullsized":"2024-04-24T18:09:22.333378+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":602,"log_dups_size":0,"ondisk_log_size":602,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0022942420000000002,"stat_sum":{"num_bytes":34481159,"num_objects":99,"num_object_clones":13,"num_object_copies":198,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":99,"num_whiteouts":68,"num_read":6059,"num_read_kb":18027,"num_write":633,"num_write_kb":12791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.a","version":"122'639","reported_seq":3642,"reported_epoch":122,"state":"active+clean","last_fresh":"2024-04-24T18:09:21.450214+0000","last_change":"2024-04-24T18:09:21.345209+0000","last_active":"2024-04-24T18:09:21.450214+0000","last_peered":"2024-04-24T18:09:21.450214+0000","last_clean":"2024-04-24T18:09:21.450214+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:09:21.450214+0000","last_undegraded":"2024-04-24T18:09:21.450214+0000","last_fullsized":"2024-04-24T18:09:21.450214+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":639,"log_dups_size":0,"ondisk_log_size":639,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00064287600000000002,"stat_sum":{"num_bytes":14747015,"num_objects":75,"num_object_clones":5,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":58,"num_read":3732,"num_read_kb":11537,"num_write":717,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"28","length":"b"}]},{"pgid":"4.b","version":"123'603","reported_seq":5286,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.379488+0000","last_change":"2024-04-24T18:09:22.306259+0000","last_active":"2024-04-24T18:09:22.379488+0000","last_peered":"2024-04-24T18:09:22.379488+0000","last_clean":"2024-04-24T18:09:22.379488+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:09:22.379488+0000","last_undegraded":"2024-04-24T18:09:22.379488+0000","last_fullsized":"2024-04-24T18:09:22.379488+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000820442,"stat_sum":{"num_bytes":22043736,"num_objects":82,"num_object_clones":11,"num_object_copies":164,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":82,"num_whiteouts":59,"num_read":7726,"num_read_kb":16517,"num_write":637,"num_write_kb":13354,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.c","version":"122'515","reported_seq":6672,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.332562+0000","last_change":"2024-04-24T18:09:22.332811+0000","last_active":"2024-04-24T18:09:22.332562+0000","last_peered":"2024-04-24T18:09:22.332562+0000","last_clean":"2024-04-24T18:09:22.332562+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:09:22.332562+0000","last_undegraded":"2024-04-24T18:09:22.332562+0000","last_fullsized":"2024-04-24T18:09:22.332562+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":515,"log_dups_size":0,"ondisk_log_size":515,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001361592,"stat_sum":{"num_bytes":24665650,"num_objects":81,"num_object_clones":11,"num_object_copies":162,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":81,"num_whiteouts":59,"num_read":7293,"num_read_kb":28455,"num_write":543,"num_write_kb":28659,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.d","version":"123'559","reported_seq":8917,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.333789+0000","last_change":"2024-04-24T18:09:22.333789+0000","last_active":"2024-04-24T18:09:22.333789+0000","last_peered":"2024-04-24T18:09:22.333789+0000","last_clean":"2024-04-24T18:09:22.333789+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:09:22.333789+0000","last_undegraded":"2024-04-24T18:09:22.333789+0000","last_fullsized":"2024-04-24T18:09:22.333789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020913059999999998,"stat_sum":{"num_bytes":32694003,"num_objects":115,"num_object_clones":20,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":82,"num_read":8793,"num_read_kb":23795,"num_write":519,"num_write_kb":9562,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.e","version":"122'518","reported_seq":6797,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.333859+0000","last_change":"2024-04-24T18:09:22.333985+0000","last_active":"2024-04-24T18:09:22.333859+0000","last_peered":"2024-04-24T18:09:22.333859+0000","last_clean":"2024-04-24T18:09:22.333859+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:09:22.333859+0000","last_undegraded":"2024-04-24T18:09:22.333859+0000","last_fullsized":"2024-04-24T18:09:22.333859+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":518,"log_dups_size":0,"ondisk_log_size":518,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0020446930000000002,"stat_sum":{"num_bytes":26474926,"num_objects":87,"num_object_clones":17,"num_object_copies":174,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":87,"num_whiteouts":58,"num_read":6601,"num_read_kb":16217,"num_write":532,"num_write_kb":13536,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.f","version":"122'777","reported_seq":5947,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305971+0000","last_change":"2024-04-24T18:09:22.306222+0000","last_active":"2024-04-24T18:09:22.305971+0000","last_peered":"2024-04-24T18:09:22.305971+0000","last_clean":"2024-04-24T18:09:22.305971+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:09:22.305971+0000","last_undegraded":"2024-04-24T18:09:22.305971+0000","last_fullsized":"2024-04-24T18:09:22.305971+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00044597899999999999,"stat_sum":{"num_bytes":20023412,"num_objects":92,"num_object_clones":10,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":65,"num_read":7901,"num_read_kb":20503,"num_write":860,"num_write_kb":8614,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.10","version":"122'592","reported_seq":7085,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.310293+0000","last_change":"2024-04-24T18:09:22.310654+0000","last_active":"2024-04-24T18:09:22.310293+0000","last_peered":"2024-04-24T18:09:22.310293+0000","last_clean":"2024-04-24T18:09:22.310293+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:09:22.310293+0000","last_undegraded":"2024-04-24T18:09:22.310293+0000","last_fullsized":"2024-04-24T18:09:22.310293+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":592,"log_dups_size":0,"ondisk_log_size":592,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00061278299999999999,"stat_sum":{"num_bytes":19642892,"num_objects":105,"num_object_clones":20,"num_object_copies":210,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":105,"num_whiteouts":76,"num_read":7841,"num_read_kb":18286,"num_write":636,"num_write_kb":14132,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.11","version":"122'675","reported_seq":8818,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.312106+0000","last_change":"2024-04-24T18:09:22.312537+0000","last_active":"2024-04-24T18:09:22.312106+0000","last_peered":"2024-04-24T18:09:22.312106+0000","last_clean":"2024-04-24T18:09:22.312106+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:09:22.312106+0000","last_undegraded":"2024-04-24T18:09:22.312106+0000","last_fullsized":"2024-04-24T18:09:22.312106+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":675,"log_dups_size":0,"ondisk_log_size":675,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0024990139999999999,"stat_sum":{"num_bytes":80803197,"num_objects":116,"num_object_clones":22,"num_object_copies":232,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":116,"num_whiteouts":80,"num_read":9362,"num_read_kb":48622,"num_write":822,"num_write_kb":30856,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.12","version":"122'572","reported_seq":5976,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305744+0000","last_change":"2024-04-24T18:09:22.306450+0000","last_active":"2024-04-24T18:09:22.305744+0000","last_peered":"2024-04-24T18:09:22.305744+0000","last_clean":"2024-04-24T18:09:22.305744+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:09:22.305744+0000","last_undegraded":"2024-04-24T18:09:22.305744+0000","last_fullsized":"2024-04-24T18:09:22.305744+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":572,"log_dups_size":0,"ondisk_log_size":572,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012581599999999999,"stat_sum":{"num_bytes":24440399,"num_objects":115,"num_object_clones":15,"num_object_copies":230,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":115,"num_whiteouts":82,"num_read":9712,"num_read_kb":31145,"num_write":536,"num_write_kb":8699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.13","version":"123'920","reported_seq":8884,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.335782+0000","last_change":"2024-04-24T18:09:22.335959+0000","last_active":"2024-04-24T18:09:22.335782+0000","last_peered":"2024-04-24T18:09:22.335782+0000","last_clean":"2024-04-24T18:09:22.335782+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:09:22.335782+0000","last_undegraded":"2024-04-24T18:09:22.335782+0000","last_fullsized":"2024-04-24T18:09:22.335782+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":920,"log_dups_size":0,"ondisk_log_size":920,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0038216650000000001,"stat_sum":{"num_bytes":19146518,"num_objects":91,"num_object_clones":19,"num_object_copies":182,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":91,"num_whiteouts":63,"num_read":10390,"num_read_kb":22682,"num_write":1282,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.14","version":"123'578","reported_seq":7476,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.381889+0000","last_change":"2024-04-24T18:09:22.312966+0000","last_active":"2024-04-24T18:09:22.381889+0000","last_peered":"2024-04-24T18:09:22.381889+0000","last_clean":"2024-04-24T18:09:22.381889+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:09:22.381889+0000","last_undegraded":"2024-04-24T18:09:22.381889+0000","last_fullsized":"2024-04-24T18:09:22.381889+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":578,"log_dups_size":0,"ondisk_log_size":578,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026239990000000001,"stat_sum":{"num_bytes":36034936,"num_objects":80,"num_object_clones":15,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":53,"num_read":9299,"num_read_kb":25220,"num_write":507,"num_write_kb":9834,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.15","version":"122'566","reported_seq":5632,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305511+0000","last_change":"2024-04-24T18:09:22.305683+0000","last_active":"2024-04-24T18:09:22.305511+0000","last_peered":"2024-04-24T18:09:22.305511+0000","last_clean":"2024-04-24T18:09:22.305511+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:09:22.305511+0000","last_undegraded":"2024-04-24T18:09:22.305511+0000","last_fullsized":"2024-04-24T18:09:22.305511+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000327015,"stat_sum":{"num_bytes":6370142,"num_objects":84,"num_object_clones":13,"num_object_copies":168,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":84,"num_whiteouts":63,"num_read":7237,"num_read_kb":17222,"num_write":474,"num_write_kb":9146,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"27","length":"7"}]},{"pgid":"4.16","version":"122'1080","reported_seq":6259,"reported_epoch":123,"state":"active+clean","last_fresh":"2024-04-24T18:09:22.305736+0000","last_change":"2024-04-24T18:09:22.305932+0000","last_active":"2024-04-24T18:09:22.305736+0000","last_peered":"2024-04-24T18:09:22.305736+0000","last_clean":"2024-04-24T18:09:22.305736+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:09:22.305736+0000","last_undegraded":"2024-04-24T18:09:22.305736+0000","last_fullsized":"2024-04-24T18:09:22.305736+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1080,"log_dups_size":0,"ondisk_log_size":1080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00071456099999999995,"stat_sum":{"num_bytes":42141907,"num_objects":109,"num_object_clones":18,"num_object_copies":218,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":109,"num_whiteouts":77,"num_read":8132,"num_read_kb":32618,"num_write":1444,"num_write_kb":16906,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"27","length":"7"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":952222507,"num_objects":3179,"num_object_clones":519,"num_object_copies":6358,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3179,"num_whiteouts":2218,"num_read":282034,"num_read_kb":852394,"num_write":27449,"num_write_kb":531286,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":39,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":55525376,"data_stored":559769443,"data_compressed":11347625,"data_compressed_allocated":36212736,"data_compressed_original":543378078,"omap_allocated":9738,"internal_metadata":0},"log_size":23427,"ondisk_log_size":23427,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308356,"kb_used_data":6904,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94063484,"statfs":{"total":96636764160,"available":96321007616,"internally_reserved":0,"allocated":7069696,"data_stored":60059913,"data_compressed":945105,"data_compressed_allocated":3694592,"data_compressed_original":57833704,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.45900000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.47399999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":0.52100000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.55100000000000005,"15min":0.503},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.6740000000000004,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.55300000000000005,"15min":0.51100000000000001},"min":{"1min":0.27300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.3879999999999999,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.751,"5min":0.53500000000000003,"15min":0.499},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":7.4749999999999996,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.41399999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.24099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.311},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.498,"15min":0.502},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.39200000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.35099999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.878,"5min":0.878,"15min":0.878},"last":0.44600000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893127,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309660,"kb_used_data":6928,"kb_used_omap":31,"kb_used_meta":302688,"kb_avail":94062180,"statfs":{"total":96636764160,"available":96319672320,"internally_reserved":0,"allocated":7094272,"data_stored":57840122,"data_compressed":1327608,"data_compressed_allocated":3989504,"data_compressed_original":55713461,"omap_allocated":32737,"internal_metadata":309952543},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.25}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27800000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.311}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39200000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36699999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38400000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":308932,"kb_used_data":11568,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94062908,"statfs":{"total":96636764160,"available":96320417792,"internally_reserved":0,"allocated":11845632,"data_stored":83219001,"data_compressed":1774201,"data_compressed_allocated":5709824,"data_compressed_original":78704524,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.59999999999999998,"15min":0.64000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.036,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.377,"5min":1.0700000000000001,"15min":1.1859999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.70099999999999996,"5min":14.15,"15min":14.15},"last":0.432}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.499,"15min":0.505},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1799999999999999,"5min":1.306,"15min":1.306},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.46800000000000003,"5min":1.135,"15min":1.246},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.2010000000000001,"5min":14.118,"15min":14.118},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":1.1000000000000001,"15min":1.208},"min":{"1min":0.34699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.65900000000000003,"5min":14.089,"15min":14.089},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.65700000000000003,"15min":0.69799999999999995},"min":{"1min":0.32500000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60899999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.49099999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.59899999999999998,"15min":0.61899999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96699999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.622,"5min":1.23,"15min":1.331},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.8240000000000001,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.58499999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.73599999999999999,"15min":0.77900000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.032,"5min":1.49,"15min":1.49},"last":0.53100000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:17 2024","interfaces":[{"interface":"back","average":{"1min":1.2070000000000001,"5min":1.2070000000000001,"15min":1.2070000000000001},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":7.8170000000000002,"5min":7.8170000000000002,"15min":7.8170000000000002},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":1.2270000000000001,"15min":1.345},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84799999999999998,"5min":14.051,"15min":14.051},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.79700000000000004,"15min":0.84099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.141,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574876,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303100,"kb_used_data":5744,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94068740,"statfs":{"total":96636764160,"available":96326389760,"internally_reserved":0,"allocated":5881856,"data_stored":51121180,"data_compressed":916961,"data_compressed_allocated":3117056,"data_compressed_original":49313082,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.40999999999999998,"15min":0.42099999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.67000000000000004,"5min":1.375,"15min":1.375},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.375,"5min":1.5389999999999999,"15min":1.7330000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54700000000000004,"5min":26.936,"15min":26.936},"last":0.27200000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":1.0960000000000001,"15min":1.206},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0629999999999999,"5min":11.93,"15min":11.93},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.40899999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":6.4889999999999999,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.29199999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.95899999999999996,"15min":1.036},"min":{"1min":0.32200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.5880000000000001,"5min":14.371,"15min":14.371},"last":0.371},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.30099999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.6390000000000002,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.38300000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":1.0580000000000001,"15min":1.1579999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":14.414,"15min":14.414},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.74099999999999999,"5min":1.329,"15min":1.329},"last":0.42199999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":6.5629999999999997,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.0920000000000001,"15min":1.173},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.52,"5min":14.452,"15min":14.452},"last":0.56000000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.60699999999999998,"5min":0.60699999999999998,"15min":0.60699999999999998},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.409,"5min":1.409,"15min":1.409},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":5.9560000000000004,"5min":5.9560000000000004,"15min":5.9560000000000004},"last":0.46600000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.82799999999999996,"15min":0.82299999999999995},"min":{"1min":0.371,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.5300000000000002,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.624,"5min":1.1080000000000001,"15min":1.1879999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.5580000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.59299999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313420,"kb_used_data":3764,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94058420,"statfs":{"total":96636764160,"available":96315822080,"internally_reserved":0,"allocated":3854336,"data_stored":19766273,"data_compressed":452614,"data_compressed_allocated":1462272,"data_compressed_original":18222740,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.443,"15min":0.41999999999999998},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.26300000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.42199999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.29699999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.40999999999999998},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.8929999999999998,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.40400000000000003,"15min":0.39200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.33000000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74299999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.39500000000000002},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.52900000000000003,"15min":0.48699999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":4.1399999999999997,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.47499999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.60999999999999999,"15min":0.57799999999999996},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.627},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":1.155,"15min":1.155},"last":0.56799999999999995}]},{"osd":5,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.372},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.0679999999999996,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.125,"5min":1.125,"15min":1.125},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.72899999999999998,"15min":0.65900000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":10.516,"5min":10.516,"15min":10.516},"last":0.51900000000000002},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.64000000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574876,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309988,"kb_used_data":12628,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94061852,"statfs":{"total":96636764160,"available":96319336448,"internally_reserved":0,"allocated":12931072,"data_stored":123320821,"data_compressed":2137338,"data_compressed_allocated":7512064,"data_compressed_original":119016401,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.37,"15min":0.35799999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.219},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28399999999999997}]},{"osd":1,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.41699999999999998,"15min":0.42599999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.2829999999999999,"5min":1.792,"15min":1.792},"last":0.254},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.30099999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.377},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"min":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6699999999999999,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.38400000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.41499999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.48799999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.53800000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.63200000000000001,"15min":0.59799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.441},{"interface":"front","average":{"1min":1.3100000000000001,"5min":0.75,"15min":0.65600000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":15.4,"5min":15.4,"15min":15.4},"last":0.46400000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":315620,"kb_used_data":10068,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94056220,"statfs":{"total":96636764160,"available":96313569280,"internally_reserved":0,"allocated":10309632,"data_stored":101470791,"data_compressed":2325730,"data_compressed_allocated":6209536,"data_compressed_original":98340115,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.746,"5min":0.746,"15min":0.746},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.72899999999999998,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.441}]},{"osd":1,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.504,"15min":0.504},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.3260000000000001,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":0.627,"5min":0.627,"15min":0.627},"last":0.42799999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.40500000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.496,"5min":0.496,"15min":0.496},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.38200000000000001}]},{"osd":5,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.26500000000000001,"5min":0.26500000000000001,"15min":0.26500000000000001},"max":{"1min":1.004,"5min":1.004,"15min":1.004},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.93400000000000005,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.35899999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.33200000000000002},"max":{"1min":0.91100000000000003,"5min":0.91100000000000003,"15min":0.91100000000000003},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.52300000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":1.371,"5min":1.371,"15min":1.371},"last":0.46999999999999997}]},{"osd":7,"last update":"Wed Apr 24 18:08:18 2024","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.88400000000000001,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.51200000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574876,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309856,"kb_used_data":8396,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94061984,"statfs":{"total":96636764160,"available":96319471616,"internally_reserved":0,"allocated":8597504,"data_stored":70895316,"data_compressed":1498968,"data_compressed_allocated":4595712,"data_compressed_original":67995331,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.64600000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.45500000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.436,"15min":0.44},"min":{"1min":0.30499999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66600000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.504},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48799999999999999,"15min":0.495},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59499999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.39500000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.52500000000000002,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.52700000000000002,"5min":1.296,"15min":1.296},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71999999999999997,"5min":1.377,"15min":1.377},"last":0.371}]},{"osd":3,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.68700000000000006,"5min":1.899,"15min":1.899},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.628,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.881,"15min":1.881},"last":0.40600000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49099999999999999,"15min":0.51000000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52800000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.35899999999999999},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.74299999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46899999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:08:20 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.32500000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.54900000000000004,"5min":1.016,"15min":1.016},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81200000000000006,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48899999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4419584,"data_stored":50390578,"data_compressed":916961,"data_compressed_allocated":3117056,"data_compressed_original":49313082,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":10244096,"data_stored":81598593,"data_compressed":1763503,"data_compressed_allocated":5672960,"data_compressed_original":77918092,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":2392064,"data_stored":18981287,"data_compressed":452614,"data_compressed_allocated":1462272,"data_compressed_original":18222740,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":11382784,"data_stored":122458937,"data_compressed":2137076,"data_compressed_allocated":7507968,"data_compressed_original":118967249,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8843264,"data_stored":100384544,"data_compressed":2314454,"data_compressed_allocated":6209536,"data_compressed_original":97987859,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":7122944,"data_stored":70158044,"data_compressed":1498968,"data_compressed_allocated":4595712,"data_compressed_original":67995331,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":5566464,"data_stored":57058434,"data_compressed":1327608,"data_compressed_allocated":3989504,"data_compressed_original":55713461,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":5554176,"data_stored":58739026,"data_compressed":936441,"data_compressed_allocated":3657728,"data_compressed_original":57260264,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:09:25.145 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-24T18:09:25.145 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:25.145 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-04-24T18:09:25.145 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-24T18:09:25.145 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-24T18:09:25.145 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-04-24T18:09:25.148 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:25.235 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:25.243+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:25.272 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:25.272 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-24T18:09:25.272 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:25.301 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:25.310+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:25.316 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:25.327 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:25.341 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:09:25.402 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:25.411+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:25.496 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:25.496 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:25.502 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:25.511+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:25.506 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:25.603 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:25.612+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:25.671 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:25.671 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:25.681 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:25.705 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:25.713+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:25.805 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:25.813+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:25.845 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:25.845 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:25.855 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:25.940 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:25.948+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:25.963 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:26.007 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:26.016+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:26.019 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:26.019 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:26.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:26.108 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:26.113 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:26.122+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:26.126 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:26.144 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:26.163 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:26.182 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:26.194 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:26.194 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:26.200 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:26.204 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:26.218 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:26.237 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:26.238 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:26.239 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:26.245 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:26.254+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:26.310 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:26.319+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:26.379 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.769843177, 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.380 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.769818762, 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.387 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:26.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.769795613, 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.392 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.393 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.394 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:26.395 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.396 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.769715045, 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.397 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.398 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.399 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.400 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:26.401 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.769690235, 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.402 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.403 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:26.404 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.405 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.406 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.769670123, 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.407 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.408 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:26.409 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:26.410 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.411 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.76959945999999, 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.412 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.413 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.414 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.415 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.416 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.769587687, 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:26.417 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.418 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.419 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.420 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.421 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.422 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.349265858, 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:26.423 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.424 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:26.425 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:26.426 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.427 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.34921867600001, 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.428 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:26.429 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:26.430 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.431 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.432 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.349193555, 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.433 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.434 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:26.435 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:26.436 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:26.437 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:26.438 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.349141421, 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.439 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.440 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:26.441 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.442 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.443 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.34907795700001, 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.444 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.445 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:26.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:26.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.575579239999996, 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.574473170999994, 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:26.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:26.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 54.419301453000003, 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.457 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:26.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:26.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.357347509, 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.356060425999999, 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:26.469 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.016884354999998, 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.473 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 26.016794660999999, 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:26.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:26.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:26.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.478 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.478 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:26.479 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:26.480 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:26.488+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:26.580 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:26.589+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:26.643 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:26.643 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:26.644 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:26.653 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:26.682 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:26.690+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:26.783 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:26.791+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:26.816 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:26.817 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:26.817 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:26.826 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:26.910 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:26.919+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:26.984 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:26.992+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:26.992 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.992 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.480982826000002, 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:09:26.996 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.480961891, 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:26.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:26.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:26.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.002 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.480932133, 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477427638999998, 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:09:27.010 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477403304999999, 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.011 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:09:27.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477381715, 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.015 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:27.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:09:27.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:09:27.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477365589000001, 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.019 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477350695999998, 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.023 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:09:27.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:09:27.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477095930000001, 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.027 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:27.031 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477068135, 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:09:27.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477035870000002, 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.036 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.477002945999999, 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:27.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:09:27.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.044 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475087847000001, 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.048 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475077839000001, 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.049 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:27.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475064392, 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.053 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475055759, 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.057 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:09:27.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:09:27.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475047262, 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:09:27.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.064 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.065 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475039626000001, 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475031972, 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:27.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:09:27.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:09:27.073 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 27.475015421999998, 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.074 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:27.075 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:27.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:09:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:09:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:27.078 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:27.078 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:27.079 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:27.121 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:27.129+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:27.187 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:27.195+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:27.219 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:27.219 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:27.220 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:27.220 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:27.230 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:27.287 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:27.296+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:27.355 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:09:27.373 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:27.374 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:27.374 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:27.374 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:27.374 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:27.388 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:27.389 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:27.396+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:27.489 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:27.498+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:27.563 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:27.563 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:27.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 94.865336448999997, 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 90.952861643000006, 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.569 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:27.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525393119999997, 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.575 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:09:27.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525371835000001, 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.579 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.582 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525354471999997, 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.585 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.586 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525309079000003, 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:09:27.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525281577999998, 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.590 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:27.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:27.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.593 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.594 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.594 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.594 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:27.594 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525263017999997, 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:27.595 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:09:27.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525239130000003, 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525221236, 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.602 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:09:27.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:09:27.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525186593999997, 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:09:27.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525092065999999, 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:09:27.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:09:27.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525073384999999, 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:09:27.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:27.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:09:27.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:09:27.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.525017906000002, 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:09:27.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:09:27.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:27.624 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.524983519999999, 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:09:27.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.524961560999998, 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.524944478000002, 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:09:27.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:09:27.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.524928017999997, 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:09:27.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.524854103000003, 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:27.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:09:27.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:09:27.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 51.524801580000002, 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:09:27.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:27.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:27.648 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:27.648 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:27.649 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:27.649 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:27.658+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:27.750 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:27.758+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:27.817 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:27.818 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:27.818 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:27.818 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:27.828 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:27.871 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:27.880+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:27.951 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:27.960+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:27.955 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:27.955 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:27.955 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:27.956 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:27.956 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:27.966 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:28.053 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:28.061+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:28.135 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:28.136 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:28.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:28.136 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:28.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.136 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.136 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671518132999999, 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.137 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:28.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:09:28.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:09:28.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:09:28.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:09:28.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:09:28.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:09:28.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:28.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:09:28.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:09:28.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:09:28.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:09:28.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:09:28.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:09:28.144 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.144 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.144 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.145 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.145 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:28.145 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671499928999999, 2024-04-24T18:09:28.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:09:28.145 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.145 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.146 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.146 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.146 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:09:28.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.146 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:09:28.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671478514, 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.152 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:28.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:09:28.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:09:28.156 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671461362999999, 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.157 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:28.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:09:28.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:09:28.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:09:28.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671445186, 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671427348999998, 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.167 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671396745999999, 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.171 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:09:28.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:09:28.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671376586999997, 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:09:28.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671360309999997, 2024-04-24T18:09:28.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:28.183 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671342719000002, 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:09:28.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.187 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.67132685, 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:28.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.190 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671308304, 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.194 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671292260000001, 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:28.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:09:28.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:09:28.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.199 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671274967999999, 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:09:28.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.203 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671253311999997, 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:09:28.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:09:28.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671234482000003, 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.207 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:09:28.210 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671218146999998, 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:09:28.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:09:28.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671202321999999, 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:28.218 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671183898000002, 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 61.671167523000001, 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:28.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.225 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.226 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.226 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:28.226 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:28.226 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:28.227 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:28.229 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:28.236+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:28.250 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:28.328 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:28.337+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:28.353 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:09:28.428 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:28.428 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:28.428 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:28.428 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:28.429 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:28.438+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:28.440 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:28.444 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:28.466 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:28.486 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:28.508 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:28.530 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:28.539+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:28.545 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:28.566 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:28.584 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:28.594 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:28.594 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:28.603 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:28.605 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:28.605 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:28.605 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:28.605 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:28.605 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:28.606 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:28.606 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:28.606 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:28.606 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:28.658 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:28.666+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:28.732 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:28.740+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:28.784 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:28.784 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:28.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:28.833 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:28.841+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:28.923 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:28.923 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:28.933 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:28.942+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:28.934 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:29.034 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:29.042+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:29.102 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:29.102 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:29.113 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:29.157 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:29.166+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:29.236 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:29.244+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:29.241 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:29.242 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:29.254 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:29.336 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:29.345+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:29.407 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:29.409 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:29.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:29.460 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:29.468+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:29.538 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:29.547+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:29.584 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:29.584 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:29.594 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:29.639 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:29.648+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:29.717 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:29.717 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:29.727 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:29.772 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:29.780+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:29.841 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:29.850+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:29.850 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:29.851 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:29.861 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:29.942 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:29.951+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:30.024 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:30.024 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:30.034 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:30.077 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:30.086+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:30.145 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:30.154+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:30.158 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:30.168 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:30.246 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:30.255+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:30.284 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:30.284 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:30.284 INFO:tasks.thrashosds.thrasher:inject_pause on osd.4 2024-04-24T18:09:30.284 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-24T18:09:30.284 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-24T18:09:30.284 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok config set filestore_inject_stall 3 2024-04-24T18:09:30.335 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:30.336 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:30.347 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:30.348 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:30.356+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:30.375 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:09:30.425 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:30.425 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-24T18:09:30.425 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:30.448 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:30.457+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:30.493 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:30.493 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:30.504 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:30.549 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:30.558+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:30.617 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:30.628 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:30.628 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:30.637 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:30.661 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:30.670+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:30.750 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:30.759+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:30.761 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:30.762 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:30.763 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:30.772 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:30.782 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:30.800 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:30.819 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:30.838 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:30.852 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:30.860+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:30.857 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:30.876 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:30.896 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:30.897 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:30.898 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:30.898 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:30.898 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:30.898 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:30.898 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:30.907 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:30.952 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:30.961+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:31.033 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:31.033 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:31.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:31.033 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.521956960000001, 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.034 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.035 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.035 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:09:31.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.035 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:09:31.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.039 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.521936024999999, 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:09:31.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.044 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.521906266999999, 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:09:31.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:09:31.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.049 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518401773000001, 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:09:31.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.054 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518377438999998, 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518355848999999, 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:09:31.059 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518339723, 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.064 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.065 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:31.069 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518324830000001, 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.070 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:09:31.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.074 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518070064, 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.075 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:09:31.079 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518042268999999, 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.080 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:09:31.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:09:31.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.518010004000001, 2024-04-24T18:09:31.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:31.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:09:31.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.090 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.517977080000001, 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:31.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:31.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.516061981, 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:09:31.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.516051973, 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.102 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.516038525999999, 2024-04-24T18:09:31.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:31.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:09:31.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.516029892999999, 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.111 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.115 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.516021395999999, 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:09:31.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.118 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.51601376, 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:09:31.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:09:31.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.516006105999999, 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.125 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:09:31.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:09:31.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 31.515989556000001, 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:31.130 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:31.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:31.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.134 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:31.135 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:31.135 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:31.136 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:31.144+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:31.137 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:31.236 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:31.244+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:31.266 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:31.267 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:31.267 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:31.267 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:31.276 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:31.361 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:31.369+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:31.370 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:09:31.438 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:31.446+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:31.459 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:31.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:31.459 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:31.459 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:31.459 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:31.470 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:31.539 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:31.547+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:31.640 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:31.648+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 98.942732887000005, 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:31.641 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:31.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:31.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:09:31.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.646 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 95.030258080999999, 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:09:31.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:09:31.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602789557999998, 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602768273000002, 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:09:31.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.660 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602750909999997, 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.661 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:09:31.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:09:31.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.664 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602705516999997, 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:09:31.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602678015999999, 2024-04-24T18:09:31.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602659455999998, 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:09:31.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:09:31.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602635567999997, 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:09:31.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:09:31.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.679 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602617674000001, 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:09:31.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602583031999998, 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:09:31.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602488504, 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.687 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602469823, 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:09:31.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602414344000003, 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:31.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:09:31.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602379958, 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.699 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:31.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:09:31.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602357998999999, 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:31.706 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602340916000003, 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:09:31.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602324455999998, 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:09:31.710 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:09:31.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:09:31.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602250540999997, 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.714 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:09:31.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.717 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 55.602198018000003, 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:31.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:31.721 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:31.722 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:31.766 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:31.774+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:31.842 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:31.850+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:31.851 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:31.851 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:31.851 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:31.851 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:31.862 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:31.946 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:31.954+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:32.030 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:32.031 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:32.031 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:32.031 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:32.031 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:32.041 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:32.082 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:32.090+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:32.145 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:32.154+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:32.211 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:32.211 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:32.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:32.211 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:32.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.211 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.747071607999999, 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.214 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.215 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.747053403999999, 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.747031989000007, 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.220 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:09:32.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:09:32.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.747014837999998, 2024-04-24T18:09:32.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:32.226 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:09:32.227 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.228 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746998661000006, 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:32.229 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.230 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.231 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.232 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746980824000005, 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.233 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.234 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:09:32.235 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:09:32.236 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746950221000006, 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:09:32.237 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.238 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.239 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.240 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746930062000004, 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.241 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:32.242 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:09:32.243 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:09:32.244 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746913785000004, 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:09:32.245 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.246 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.247 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.248 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746896194000001, 2024-04-24T18:09:32.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.253 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746880325000006, 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:32.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:09:32.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:09:32.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746861779, 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746845734999994, 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.265 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746828442999998, 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:09:32.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:32.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:09:32.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:09:32.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746806786999997, 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:09:32.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746787956999995, 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:09:32.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:32.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:09:32.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746771621999997, 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.278 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.281 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746755797000006, 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:32.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:09:32.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746737373000002, 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:32.285 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:32.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:09:32.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:09:32.288 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 65.746720998000001, 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:09:32.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.292 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.293 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:32.294 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:32.294 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:32.302+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:32.395 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:32.403+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:32.472 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:32.472 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:32.472 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:32.472 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:32.482 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:32.496 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:32.504+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:32.597 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:32.605+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:32.651 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:32.651 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:32.651 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:32.652 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:32.652 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:32.662 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:32.703 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:32.711+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:32.799 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:32.807+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:32.832 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703126021000003, 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.833 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:09:32.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:09:32.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703111480999993, 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.837 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:32.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:09:32.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703096630000005, 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.841 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.842 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.843 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.844 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703081480999998, 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.845 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.846 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:09:32.847 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:09:32.848 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:32.849 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703066695000004, 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.850 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:09:32.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:09:32.853 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703052106000001, 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.854 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:09:32.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:09:32.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703029280999999, 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:09:32.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.862 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.703013627000004, 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:32.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.702998504999996, 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:09:32.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.702982688999995, 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:32.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:09:32.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:09:32.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.702967686999997, 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:09:32.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:32.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:09:32.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.702952823000004, 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.879 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:09:32.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.702931706000001, 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.883 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:32.887 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.702916475999999, 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:09:32.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:09:32.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.702901738999998, 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.893 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.895 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.697800529000006, 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.896 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:32.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:09:32.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:09:32.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.697764683000003, 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.900 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:09:32.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:09:32.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.697744675999999, 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.904 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:32.908 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.695923484999994, 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:09:32.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 80.695898205000006, 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:32.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.916 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.917 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.917 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:32.917 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:32.917 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:32.917 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:32.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:32.919 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:32.926+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:33.019 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:33.026+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:33.093 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:33.093 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:33.094 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:33.094 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:33.104 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:33.107 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:33.125 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:33.126 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:33.133+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:33.145 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:33.162 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:33.181 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:33.201 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:33.220 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:33.220 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:33.228+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:33.235 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:33.235 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:33.239 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:33.240 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:33.240 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:33.240 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:33.240 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:33.241 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:33.246 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:33.321 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:33.330+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:33.384 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:09:33.428 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:33.436+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:33.433 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:33.433 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:33.445 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:33.524 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:33.532+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:33.578 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:33.578 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:33.588 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:33.625 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:33.633+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:33.711 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:33.711 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:33.721 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:33.725 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:33.733+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:33.826 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:33.834+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:33.844 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:33.844 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:33.854 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:33.926 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:33.935+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:33.979 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:33.979 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:33.988 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:34.028 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:34.036+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:34.129 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:34.137+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:34.152 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:34.152 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:34.162 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:34.230 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:34.238+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:34.326 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:34.326 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:34.331 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:34.340+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:34.336 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:34.376 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:09:34.432 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:34.440+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:34.500 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:34.501 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:34.510 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:34.552 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:34.560+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:34.634 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:34.642+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:34.675 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:34.675 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:34.685 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:34.735 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:34.744+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:34.809 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:34.809 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:34.819 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:34.861 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:34.869+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:34.937 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:34.946+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:34.983 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:34.983 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:34.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:35.078 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:35.086+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:35.139 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:35.147+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:35.157 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:35.157 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:35.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:35.240 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:35.248+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:35.253 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:35.331 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:35.331 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:35.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:35.342 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:35.349+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:35.416 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:35.435 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:35.437 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:35.437 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:35.438 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-04-24T18:09:35.438 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-24T18:09:35.438 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-24T18:09:35.438 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-04-24T18:09:35.454 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:35.472 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:35.478 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:35.478 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:35.479 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:35.479 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:35.482 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:35.491+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:35.489 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:35.491 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:35.512 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:35.532 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:35.543 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:35.551+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:35.551 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:35.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:35.563 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:35.564 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "" 2024-04-24T18:09:35.564 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:35.621 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:35.621 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:35.621 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:35.621 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:35.621 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:35.631 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:35.644 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:35.652+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:35.744 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:35.753+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.063670223, 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:35.762 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.763 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:09:35.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:09:35.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:09:35.766 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.767 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 99.151195416999997, 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:09:35.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:09:35.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.772 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723726894000002, 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:09:35.774 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:35.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.776 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723705609, 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:35.777 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:35.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:09:35.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723688246000002, 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.780 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:09:35.781 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:09:35.782 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:09:35.783 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723642853000001, 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.784 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:09:35.785 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.786 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723615352000003, 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:35.788 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.789 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:09:35.790 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723596792000002, 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:09:35.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:09:35.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:09:35.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.795 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723572904000001, 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723555009999998, 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.799 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:09:35.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:09:35.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723520368000003, 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.803 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.805 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.807 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723425839999997, 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.808 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.809 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.811 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723407158999997, 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.812 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.813 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.814 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.815 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.72335168, 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.816 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.817 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.818 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723317293999997, 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:09:35.819 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.820 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.821 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.822 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723295335000003, 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.823 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.824 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:09:35.825 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:35.826 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723278252, 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.827 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.828 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.829 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723261792000002, 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.830 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.831 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:09:35.832 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.833 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723187877000001, 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.834 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:09:35.835 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:09:35.836 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 59.723135354, 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:09:35.837 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:35.838 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:09:35.839 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:09:35.840 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.841 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.841 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:35.841 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:35.841 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:35.842 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:35.845 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:35.852+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:35.945 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:35.953+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:35.971 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:35.971 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:35.971 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:35.971 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:35.981 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:36.065 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:36.073+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:36.147 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:36.156+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:36.150 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:36.150 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:36.150 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:36.150 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:36.150 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:36.160 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:36.249 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:36.257+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:36.330 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865912140000006, 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.331 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:09:36.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:09:36.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865893936000006, 2024-04-24T18:09:36.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:09:36.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865872521, 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:09:36.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865855370000006, 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.344 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:09:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.345 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865839192999999, 2024-04-24T18:09:36.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:09:36.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:09:36.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865821355999998, 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.357 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865790752999999, 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.358 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:09:36.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865770593999997, 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:09:36.366 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865754316999997, 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.367 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:09:36.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:09:36.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865736725999994, 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:09:36.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:09:36.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:09:36.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:09:36.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865720856999999, 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:36.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:09:36.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865702311000007, 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:09:36.380 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:36.384 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865686267000001, 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.388 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865668975000006, 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:09:36.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:09:36.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.392 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865647319000004, 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:36.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:09:36.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:09:36.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865628489000002, 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:09:36.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865612154000004, 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.403 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865596328999999, 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:09:36.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.407 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865577904999995, 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:36.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:09:36.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:09:36.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 69.865561529999994, 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:09:36.411 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:36.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.415 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:36.416 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:09:36.417 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:36.417 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:36.424+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:36.518 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:36.526+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:36.592 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:36.592 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:36.592 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:36.592 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:36.605 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:36.618 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:36.627+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:36.719 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:36.727+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:36.734 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:36.734 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:36.734 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:36.734 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:36.745 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:36.820 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:36.829+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:36.916 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786874424999993, 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.917 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786859884999998, 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:09:36.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.925 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:09:36.926 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786845033999995, 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.927 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.928 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.929 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.930 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:09:36.931 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786829885000003, 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.932 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.933 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:09:36.934 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.935 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:09:36.936 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786815098999995, 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.937 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.938 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:09:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.940 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:09:36.941 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786800510000006, 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.942 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.943 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.944 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:09:36.945 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.946 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786777685000004, 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.947 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:36.948 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.949 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.950 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:09:36.951 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786762030999995, 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.952 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:09:36.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.956 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786746909000001, 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786731093, 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:36.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:09:36.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.966 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786716091000002, 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:36.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.969 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:09:36.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786701226999995, 2024-04-24T18:09:36.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:09:36.975 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786680110000006, 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.976 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:09:36.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786664880000004, 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.981 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:09:36.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.985 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.786650143000003, 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:36.990 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.781548932999996, 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:36.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:09:36.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.781513086999993, 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:36.995 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:36.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:09:36.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:36.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.781493080000004, 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.779671888999999, 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.005 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:09:37.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:09:37.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 84.779646608999997, 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.010 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:09:37.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.014 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.015 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.015 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:37.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:37.017 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:37.024+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:37.117 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:37.125+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:37.146 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:37.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:37.146 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:37.146 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:37.156 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:37.218 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:37.226+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:37.305 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:09:37.319 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:37.327+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:37.335 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:37.336 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:37.336 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:37.336 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:37.336 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:37.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:37.420 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:37.428+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.345479608999995, 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.325840482999993, 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.522 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.325808761000005, 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.526 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:37.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:09:37.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.530 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.325785343999996, 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.534 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.325755271000006, 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.535 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:09:37.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:09:37.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:09:37.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:09:37.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:09:37.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:09:37.539 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.539 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324710124000006, 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.540 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:09:37.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:09:37.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:09:37.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:09:37.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.545 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.545 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.545 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.546 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:37.546 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324692893000005, 2024-04-24T18:09:37.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:09:37.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:09:37.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.547 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:37.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:37.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:09:37.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:09:37.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:09:37.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:09:37.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:09:37.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:09:37.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:09:37.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:09:37.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:09:37.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:09:37.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:09:37.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:09:37.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:09:37.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.554 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.555 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.555 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.555 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:37.555 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324675894999999, 2024-04-24T18:09:37.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:09:37.556 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.556 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.556 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.556 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:09:37.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.557 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:37.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.558 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:37.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:09:37.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:09:37.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:09:37.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:09:37.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:09:37.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:09:37.564 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.564 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.564 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.565 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.565 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:37.565 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324657213999998, 2024-04-24T18:09:37.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:09:37.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:09:37.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.567 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:37.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:37.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:37.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:09:37.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:09:37.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.570 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.570 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:09:37.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:09:37.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:09:37.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:09:37.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:09:37.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:09:37.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:09:37.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:09:37.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:09:37.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:09:37.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.574 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.574 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.574 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.575 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:37.575 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324639746000003, 2024-04-24T18:09:37.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:09:37.575 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.575 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.575 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.576 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.576 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.576 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:09:37.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.576 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:37.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:37.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:09:37.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:09:37.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:09:37.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:09:37.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:09:37.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.581 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:09:37.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:37.581 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.581 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.581 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:09:37.582 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:09:37.582 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.582 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.582 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.582 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:09:37.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:09:37.583 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.583 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.583 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.583 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:09:37.583 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:09:37.584 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.584 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.584 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.584 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.584 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.584 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.585 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:37.585 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324623622000004, 2024-04-24T18:09:37.585 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:09:37.585 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.585 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.585 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.586 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.586 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.586 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:09:37.586 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.586 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.586 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:37.587 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.587 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.587 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.587 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.587 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:37.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.588 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.588 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.588 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.588 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:09:37.588 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.589 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.589 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:09:37.589 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:09:37.589 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.589 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.590 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:09:37.590 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:09:37.590 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.590 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.590 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:09:37.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:09:37.591 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.591 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.591 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.591 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:09:37.591 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:09:37.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.592 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.592 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.592 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:09:37.592 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:09:37.592 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.593 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.593 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.593 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:09:37.593 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:09:37.593 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.593 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.593 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.594 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.594 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.594 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.594 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:37.594 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324592515999996, 2024-04-24T18:09:37.594 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:09:37.595 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.595 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.595 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.595 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:09:37.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.596 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.596 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.596 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.596 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:37.596 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.596 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.597 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:37.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.597 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.597 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.597 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:09:37.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:09:37.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:09:37.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:09:37.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:09:37.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:09:37.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:09:37.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:09:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:09:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.603 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.603 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:37.603 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324573314999995, 2024-04-24T18:09:37.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:09:37.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.604 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:09:37.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:09:37.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:09:37.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.324552272999995, 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.608 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.610 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.323797325000001, 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.612 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.323770823000004, 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.323753292999996, 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.322290132999996, 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.624 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.310451485000002, 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.628 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 85.310435142000003, 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:37.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:37.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:09:37.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:09:37.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.636 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:37.636 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:37.636 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:37.636 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:37.637 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:37.645+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:37.638 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:37.738 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:37.746+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:37.779 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:37.779 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:37.780 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:37.780 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:37.790 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:37.823 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:37.840 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:37.848+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:37.842 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:37.860 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:37.879 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:37.899 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:37.916 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:37.916 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:37.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:37.926 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:37.936 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:37.954 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:37.956 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:37.957 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:37.968 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:37.976+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:38.041 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:38.050+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:38.091 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:38.091 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:38.101 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:38.143 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:38.151+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:38.244 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:38.252+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:38.266 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:38.276 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:38.344 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:38.352+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:38.442 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:38.442 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:38.444 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:38.452+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:38.452 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:38.546 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:38.554+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:38.576 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:38.576 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:38.586 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:38.646 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:38.654+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:38.747 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:38.755+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:38.751 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:38.751 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:38.762 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:38.848 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:38.856+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:38.885 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:38.885 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:38.895 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:38.950 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:38.958+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:39.019 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:39.019 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:39.029 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:39.073 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:39.079+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:39.152 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:39.160+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:39.197 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:39.197 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:39.207 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:39.252 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:39.261+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:39.322 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:09:39.330 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:39.330 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:39.341 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:39.382 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:39.390+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:39.453 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:39.462+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:39.505 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:39.505 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:39.515 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:39.558 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:39.566+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:39.644 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:39.654 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:39.696 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:39.704+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:39.757 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:39.765+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:39.822 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:39.823 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:39.833 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:39.878 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:39.886+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:39.958 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:39.966+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:39.961 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:39.961 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:39.967 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:39.971 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:40.059 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:40.067+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:40.116 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:40.136 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:40.145 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:40.145 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:40.145 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:40.155 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:40.156 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:40.174 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:40.175 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:40.182+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:40.195 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:40.215 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:40.234 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:40.254 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:40.255 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:40.255 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:40.256 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:40.260 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:40.268+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:40.305 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:40.305 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:40.306 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:40.306 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:40.315 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:40.361 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:40.369+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:40.385 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:09:40.462 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:40.469+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:40.488 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024375626999998, 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:09:40.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024357422999998, 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.493 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024336008000006, 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024318856999997, 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:09:40.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:09:40.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024302680000005, 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:40.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:09:40.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:09:40.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024284843000004, 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024254240000005, 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:09:40.518 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024234081000003, 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:09:40.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:40.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:09:40.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:09:40.522 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024217804000003, 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.523 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:09:40.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.526 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024200213, 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.530 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024184344000005, 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:09:40.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024165797999999, 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.535 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:09:40.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:09:40.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024149754000007, 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:09:40.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:09:40.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024132461999997, 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.543 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:09:40.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.546 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024110805999996, 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.547 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024091975999994, 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:09:40.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024075640999996, 2024-04-24T18:09:40.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:09:40.557 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024059816000005, 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.558 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:09:40.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.561 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024041392000001, 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 74.024025017, 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:40.565 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.568 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:40.569 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:40.569 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:40.570 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:40.571 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:40.578+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:40.574 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:40.574 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:40.574 INFO:tasks.thrashosds.thrasher:inject_pause on osd.5 2024-04-24T18:09:40.574 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-24T18:09:40.574 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-24T18:09:40.575 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok config set filestore_inject_stall 3 2024-04-24T18:09:40.671 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:40.678+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:40.716 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:40.716 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-04-24T18:09:40.716 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:40.740 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:40.741 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:40.741 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:40.751 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:40.792 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:40.800+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:40.872 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:40.880+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:40.920 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:40.920 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:40.920 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:40.920 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:40.920 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:40.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:40.974 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:40.982+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:41.060 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.060 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930342534999994, 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.061 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:09:41.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930327994999999, 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.066 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:09:41.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.070 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930313143999996, 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:41.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:09:41.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.075 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930297995000004, 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:09:41.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:09:41.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.080 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930283208999995, 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:09:41.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.085 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930268620000007, 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:09:41.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930245795000005, 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:09:41.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:09:41.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.095 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930230140999996, 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.096 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:09:41.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:09:41.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930215019000002, 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.101 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:09:41.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930199203000001, 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.106 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:09:41.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:41.111 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930184201000003, 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:41.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:09:41.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930169336999995, 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:09:41.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:09:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:09:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930148220000007, 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:41.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:09:41.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.127 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930132990000004, 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:09:41.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:41.132 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.930118253000003, 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:41.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:09:41.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.925017042999997, 2024-04-24T18:09:41.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:09:41.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.924981196999994, 2024-04-24T18:09:41.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:09:41.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:09:41.146 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.924961190000005, 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:09:41.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:09:41.151 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.923139999, 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.152 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:41.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:09:41.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.156 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 88.923114718999997, 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.157 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:41.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:09:41.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.162 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.162 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.162 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:41.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:41.164 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:41.172+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:41.264 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:41.272+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:41.333 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:41.333 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:41.333 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:41.334 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:41.344 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:41.365 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:41.374+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:41.467 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:41.475+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:41.512 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:41.512 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:41.512 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:41.522 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:41.568 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:41.576+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.481870211, 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:09:41.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:09:41.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:41.657 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.462231084999999, 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:09:41.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.462199362999996, 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:41.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.462175946000002, 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:09:41.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:09:41.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.462145872999997, 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:09:41.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:09:41.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:09:41.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.461100725999998, 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.461083494999997, 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:41.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:09:41.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.680 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.461066497000004, 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.461047816000004, 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.684 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.461030347999994, 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.688 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:09:41.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:09:41.691 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.461014223999996, 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.692 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.460983118000001, 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:09:41.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:09:41.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.699 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.460963917000001, 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.460942875000001, 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:09:41.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:09:41.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.707 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.460187927000007, 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:09:41.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.460161424999995, 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:09:41.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:09:41.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.460143895000002, 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.715 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:09:41.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.458680735000002, 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.719 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.446842086999993, 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.723 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:09:41.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:09:41.726 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.446825743999995, 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.727 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.730 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.731 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:41.731 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:41.731 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:41.732 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:41.733 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:41.740+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:41.833 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:41.841+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:41.871 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:41.871 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:41.872 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:41.882 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:41.965 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:41.974+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:42.034 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:42.042+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:42.045 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:42.046 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:42.046 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:42.055 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:42.139 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:42.148+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.89023319099999, 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:42.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:42.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.89021728500001, 2024-04-24T18:09:42.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:42.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.890165318, 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.889241844, 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.889225212, 2024-04-24T18:09:42.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.888523609, 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.242 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:42.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.888507107, 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:42.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:42.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.88760424100001, 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.249 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:42.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.88759621600001, 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:42.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:42.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.256 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.886532074, 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:42.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:42.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:42.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.260 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 112.886516105, 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:42.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 75.826317665999994, 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:42.264 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:42.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:42.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:42.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:42.269 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.657293633999998, 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:42.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.657277399999998, 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.273 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:42.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:42.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.657262180000004, 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.277 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:42.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:42.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:42.280 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.657234600999999, 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:42.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.657216300000002, 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.657200001000007, 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.289 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 48.348797382000001, 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:42.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:42.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 42.999524276999999, 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:42.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:42.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.301 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:42.302 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:42.302 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:42.302 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:42.304 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:42.304 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:42.311+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:42.397 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:09:42.404 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:42.412+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:42.471 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:42.471 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:42.471 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:42.475 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:42.481 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:42.494 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:42.513 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:42.523 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:42.531+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:42.533 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:42.558 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:42.576 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:42.595 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:42.606 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:42.615+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:42.613 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:42.614 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:42.615 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:42.648 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:42.648 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:42.658 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:42.742 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:42.751+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:42.808 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:42.816+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:42.822 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:42.822 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:42.832 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:42.909 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:42.917+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:42.996 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:42.996 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:43.006 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:43.010 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:43.018+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:43.111 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:43.119+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:43.130 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:43.130 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:43.140 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:43.212 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:43.220+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:43.264 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:43.264 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:43.274 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:43.315 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:43.324+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:43.397 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:09:43.414 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:43.422+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:43.440 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:43.440 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:43.450 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:43.534 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:43.543+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:43.614 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:43.614 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:43.615 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:43.623+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:43.623 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:43.715 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:43.724+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:43.787 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:43.788 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:43.798 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:43.816 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:43.825+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:43.917 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:43.925+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:43.941 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:43.941 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:43.951 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:44.036 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:44.044+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:44.119 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:44.127+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:44.120 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:44.120 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:44.131 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:44.221 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:44.228+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:44.300 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:44.300 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:44.311 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:44.321 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:44.330+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:44.422 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:44.429+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:44.480 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:44.481 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:44.491 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:44.522 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:44.531+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:44.623 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:44.631+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:44.627 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:44.663 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:44.664 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:44.674 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:44.725 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:44.733+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:44.776 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:44.794 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:44.814 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:44.825 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:44.833+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:44.833 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:44.849 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:44.849 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385438074000007, 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.850 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:09:44.851 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.852 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:09:44.853 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385419870000007, 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:09:44.854 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.855 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.856 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.857 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385398455000001, 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.858 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.859 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.860 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.861 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385381304000006, 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:09:44.862 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:09:44.863 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.864 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.865 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385365127, 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.866 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.867 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.868 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.869 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385347289999999, 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.870 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.871 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:09:44.872 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.873 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385316687, 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.874 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.875 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:09:44.876 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.877 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385296527999998, 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.878 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:09:44.879 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.880 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385280250999998, 2024-04-24T18:09:44.881 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.882 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.883 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:09:44.884 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385262659999995, 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.885 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.886 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.887 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.888 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385246791, 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.889 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.890 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.891 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.892 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385228244999993, 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:09:44.893 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.894 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.895 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.896 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385212201000002, 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.897 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.898 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.899 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385194909000006, 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.900 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.901 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.902 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:09:44.903 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385173253000005, 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:09:44.904 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.905 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.906 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385154423000003, 2024-04-24T18:09:44.907 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.908 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.909 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:09:44.910 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385138088000005, 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:44.911 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.912 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.913 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.914 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385122263, 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.915 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.916 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:09:44.917 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385103838999996, 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:09:44.918 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.919 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.920 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 78.385087463999994, 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:09:44.921 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:44.922 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:44.923 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:09:44.924 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:44.925 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:44.925 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:44.925 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:44.927 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:44.927 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:44.927 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:44.927 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:44.928 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:44.929 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:44.934+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:45.027 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:45.035+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:45.095 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:45.095 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:45.095 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:45.095 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:45.105 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:45.127 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:45.136+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:45.229 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:45.237+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:45.273 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:45.273 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:45.273 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:45.273 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:45.273 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:45.284 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:45.330 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:45.338+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:45.410 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:09:45.431 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:45.439+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329160189000007, 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:45.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:09:45.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329145648999997, 2024-04-24T18:09:45.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:09:45.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329130797999994, 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.471 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329115649000002, 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:09:45.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:09:45.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329100862999994, 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:09:45.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:09:45.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:09:45.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329086274000005, 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329063449000003, 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:09:45.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:09:45.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329047794999994, 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:45.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:09:45.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329032673, 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:09:45.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:45.496 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329016856999999, 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.500 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.329001855000001, 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.504 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.328986990999994, 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:45.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.508 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.328965874000005, 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.509 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:09:45.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.328950644000003, 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.328935907000002, 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:45.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:09:45.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:09:45.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.323834696999995, 2024-04-24T18:09:45.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.525 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.323798851000006, 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:09:45.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:09:45.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.323778844000003, 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.321957652999998, 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.321932372999996, 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.538 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:45.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.541 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:45.542 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:45.542 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:45.543 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:45.543 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:45.551+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:45.643 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:45.652+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:45.712 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:45.713 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:45.713 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:45.713 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:45.723 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:45.728 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:45.728 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:45.729 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-04-24T18:09:45.729 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-04-24T18:09:45.729 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-04-24T18:09:45.729 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-04-24T18:09:45.773 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:45.781+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:45.845 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:45.853+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:45.853 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:45.853 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:45.853 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:45.853 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:45.853 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:45.863 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:45.866 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:45.866 INFO:teuthology.orchestra.run.smithi078.stdout: "success": "" 2024-04-24T18:09:45.867 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:45.947 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:45.955+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:46.033 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.862263585999997, 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:09:46.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.842624459999996, 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.039 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.842592737999993, 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:46.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:09:46.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:09:46.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.047 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.842569320999999, 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.048 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:09:46.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:09:46.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:46.052 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.842539247999994, 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:09:46.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:09:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:09:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:09:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:09:46.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:09:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.060 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841494100999995, 2024-04-24T18:09:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:09:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:09:46.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:09:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:09:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:46.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:46.068 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841476869999994, 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.070 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:09:46.072 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841459872000001, 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.073 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:46.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:09:46.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:09:46.076 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.077 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841441191000001, 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.078 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:09:46.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:09:46.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:09:46.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:09:46.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:09:46.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841423723000005, 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:09:46.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841407598999993, 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.088 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:09:46.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:09:46.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841376492999999, 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:09:46.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:46.098 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841357291999998, 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.099 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:09:46.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.103 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.841336249999998, 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:46.108 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.840581302000004, 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:46.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:09:46.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:09:46.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.113 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.840554800000007, 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:09:46.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:09:46.117 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.840537269999999, 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.118 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:46.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:09:46.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:09:46.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:09:46.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.839074109999999, 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.123 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:09:46.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.127 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.827235462000004, 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:09:46.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:09:46.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:09:46.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:46.132 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.827219119000006, 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.136 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:46.137 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:46.137 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:46.138 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:46.139 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:46.146+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:46.239 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:46.247+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:46.266 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:46.266 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:46.266 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:46.266 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:46.276 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:46.340 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:46.348+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:46.400 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:46.400 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:46.400 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:46.400 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:46.400 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:46.402 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:09:46.411 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:46.454 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:46.463+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:46.535 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.20384194499999, 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.536 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:46.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:46.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.20382603900001, 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:46.540 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:46.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.203774072, 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.544 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:46.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:46.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.202850598, 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:46.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.202833966, 2024-04-24T18:09:46.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:46.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:46.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:46.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.202132363, 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.555 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.556 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:46.557 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:46.558 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.202115861, 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:46.559 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.560 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.561 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:46.562 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.20121299500001, 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.563 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.564 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.565 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.566 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.20120497000001, 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.567 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:46.568 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.569 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.570 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.200140828, 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:46.571 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.572 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:46.573 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.574 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.200124859, 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.575 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.576 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.577 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.139926419999995, 2024-04-24T18:09:46.578 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.579 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.580 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.581 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.582 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.583 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.970902387999999, 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.584 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.585 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.586 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.587 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.970886153999999, 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:46.588 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:46.589 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:46.590 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:46.591 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.970870934000004, 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.592 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:46.593 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:46.594 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.595 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.970843355, 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.596 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:46.597 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:46.598 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.599 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.970825054000002, 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.600 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.601 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.602 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.603 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.970808754999993, 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.604 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:46.605 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.606 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.607 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 52.662406136000001, 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.608 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.609 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.610 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:46.611 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.612 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 47.313133031, 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.613 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.614 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:46.615 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:46.616 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:46.618 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:46.618 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:46.626+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:46.718 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:46.727+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:46.783 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:46.783 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:46.783 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:46.783 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:46.793 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:46.834 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:46.843+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:46.920 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:46.928+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:46.930 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:46.958 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:46.958 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:46.958 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:46.958 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:46.958 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:46.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:47.021 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:47.029+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:47.089 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:47.107 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:47.123 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:47.131+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:47.125 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.779697601999999, 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.136 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.779689473000005, 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:47.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:47.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.779681449999998, 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:47.144 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.779672536999996, 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:47.148 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.779663717999995, 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.156 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.779654586000007, 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:47.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.160 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778834326999998, 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.161 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:47.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:47.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778807127999997, 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.165 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778789657999994, 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.169 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:47.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:47.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778774318000004, 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.173 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778750239999994, 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.177 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778734674999995, 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.181 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:47.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778719714000005, 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.185 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:47.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778702774999999, 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:47.189 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778686370000003, 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:47.193 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:47.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778669930999996, 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:47.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778655091000005, 2024-04-24T18:09:47.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778638666999996, 2024-04-24T18:09:47.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:47.209 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778621451999996, 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:47.213 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 76.778603986999997, 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:47.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:47.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:47.217 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:47.219 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:47.219 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:47.219 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:47.219 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:47.219 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:47.220 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:47.221 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:47.261 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:47.269+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:47.324 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:47.332+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:47.384 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:47.384 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:47.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:47.425 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:47.433+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:47.517 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:47.518 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:47.526 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:47.534+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:47.527 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:47.626 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:47.635+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:47.651 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:47.651 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:47.661 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:47.727 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:47.736+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:47.785 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:47.785 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:47.794 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:47.835 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:47.844+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:47.929 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:47.938+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:47.958 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:47.958 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:47.968 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:48.051 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:48.060+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:48.131 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:48.139+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:48.131 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:48.131 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:48.142 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:48.232 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:48.240+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:48.286 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:48.286 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:48.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:48.333 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:48.341+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:48.414 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:09:48.431 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:48.431 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:48.433 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:48.442+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:48.442 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:48.534 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:48.542+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:48.618 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:48.618 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:48.629 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:48.636 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:48.644+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:48.737 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:48.745+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:48.798 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:48.798 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:48.808 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:48.838 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:48.846+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:48.937 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:48.937 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:48.938 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:48.946+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:48.947 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:49.039 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:49.047+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:49.115 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:49.115 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:49.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:49.169 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:49.177+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:49.233 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:49.240 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:49.249+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:49.254 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:49.254 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:49.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:49.341 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:49.349+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:49.380 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:49.398 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:49.420 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:09:49.426 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:49.442 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:49.450+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:49.450 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:49.450 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:49.450 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:49.453 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:49.462 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:49.476 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:49.500 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:49.528 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:49.543 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:49.551+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:49.551 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:49.552 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:49.552 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:49.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:49.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:49.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:49.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:49.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:49.553 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:49.615 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:49.615 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485856656999999, 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:09:49.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:09:49.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485842117000004, 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.620 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:09:49.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485827266000001, 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.624 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:09:49.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485812116999995, 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.628 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:09:49.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:09:49.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:09:49.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485797331000001, 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:09:49.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485782741999998, 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:09:49.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:09:49.639 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485759916999996, 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:09:49.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.643 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485744263000001, 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:09:49.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.647 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485729140999993, 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485713325000006, 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.655 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485698322999994, 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.656 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.659 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485683459000001, 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.663 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485662341999998, 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.667 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485647111999995, 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.668 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.485632374999994, 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.672 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:09:49.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.480531165000002, 2024-04-24T18:09:49.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:09:49.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:49.680 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.480495318999999, 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:49.684 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.480475311999996, 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:09:49.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:09:49.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.688 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.478654121000005, 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:09:49.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:09:49.692 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.478628841000003, 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:09:49.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:49.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:49.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:09:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:09:49.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.697 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.697 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:49.697 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:49.697 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:49.698 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:49.699 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:49.706+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:49.798 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:49.807+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:49.866 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:49.866 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:09:49.867 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:49.877 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:49.899 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:49.907+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:49.999 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:50.008+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:50.005 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:50.005 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:50.005 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:50.005 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:50.005 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:50.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:50.101 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:50.109+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:50.145 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:50.145 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.974498823999994, 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.146 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.954859698000007, 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:50.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:09:50.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.153 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.954827976000004, 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.155 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:09:50.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.954804558999996, 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.157 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:50.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:09:50.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.954774486000005, 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.161 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953729339000006, 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.165 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:09:50.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953712108000005, 2024-04-24T18:09:50.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:09:50.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:09:50.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.173 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953695109999998, 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:09:50.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953676428999998, 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:09:50.177 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.179 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.180 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:50.181 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953658961000002, 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.182 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:09:50.183 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:09:50.184 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.185 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953642837000004, 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.186 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.187 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.188 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.189 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953611730999995, 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.190 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.191 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.192 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.193 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953592529999995, 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.194 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.195 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:09:50.196 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.197 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.953571487999994, 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.198 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:50.199 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:09:50.200 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.201 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:50.202 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.952816540000001, 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.203 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.204 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.205 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.952790038000003, 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.206 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.207 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.208 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.209 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.952772507999995, 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.210 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.211 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.212 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:09:50.213 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.951309347999995, 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.214 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.215 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:09:50.216 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.217 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.939470700000001, 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.218 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.219 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:09:50.220 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.221 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.939454357000002, 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:50.222 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:50.223 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:50.224 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:50.225 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:50.227 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:50.227 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:50.235+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:50.328 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:50.336+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:50.367 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:50.367 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:50.367 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:50.367 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:50.378 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:50.429 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:50.437+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:50.514 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:50.514 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:50.514 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:50.525 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:50.529 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:50.537+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:50.630 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:50.638+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.318195884, 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.650 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.318179978, 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:50.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:50.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.318128011, 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.659 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:50.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:50.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.31720453699999, 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.663 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.317187905, 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:50.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.670 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.316486302, 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:50.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.31646979999999, 2024-04-24T18:09:50.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:50.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.315566934, 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:50.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:50.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.315558909, 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:50.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:50.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.314494767, 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:50.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:50.690 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 121.314478798, 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.694 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 84.254280359000006, 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:50.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.085256326999996, 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.700 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:50.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:50.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.085240092999996, 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:50.704 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.085224873000001, 2024-04-24T18:09:50.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:50.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:50.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.085197293999997, 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:50.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.716 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.085178993, 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:50.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.720 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 74.085162694000005, 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:50.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:50.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 56.776760074999999, 2024-04-24T18:09:50.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:50.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.728 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.427486969999997, 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.729 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:50.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:50.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:50.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:50.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.733 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.733 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:50.733 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:50.733 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:50.734 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:50.735 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:50.742+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:50.835 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:50.843+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:50.877 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:50.877 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:50.877 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-04-24T18:09:50.877 INFO:tasks.thrashosds.thrasher:Setting osd 1 primary_affinity to 0.854659 2024-04-24T18:09:50.878 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 0.8546588491191446 2024-04-24T18:09:50.898 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:50.898 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:50.898 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:50.898 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:50.908 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:50.950 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:50.958+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:51.036 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:51.044+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:51.074 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:51.074 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:51.074 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:51.074 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:51.074 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:51.084 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:51.137 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:51.146+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.853399714000005, 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.210 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:51.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:51.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.853391584999997, 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:51.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.218 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.218 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:51.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:51.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.218 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.853383562000005, 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.853374649000003, 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.223 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:51.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:51.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.853365830000001, 2024-04-24T18:09:51.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:51.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.231 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.853356697999999, 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:51.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852536439000005, 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.239 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852509240000003, 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.85249177, 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.246 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:51.247 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852476429999996, 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:51.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:51.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852452352, 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:51.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852436787000002, 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:51.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852421825999997, 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.259 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:51.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.262 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852404887000006, 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.263 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.266 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852388481999995, 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:51.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852372043000003, 2024-04-24T18:09:51.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:51.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852357202999997, 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.274 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:51.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:51.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852340779000002, 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.278 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:51.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:51.281 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852323564000002, 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:51.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 80.852306099000003, 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:51.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.289 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:51.291 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:51.291 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:51.299+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:51.391 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:51.400+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:51.424 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.1 primary-affinity to 0.854659 (8560102) 2024-04-24T18:09:51.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:09:51.458 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:51.459 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:51.459 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:51.459 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:51.469 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:51.512 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:51.521+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:51.565 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:51.598 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:51.598 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:51.598 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:51.598 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:51.598 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:51.608 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:51.609 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:51.617+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:51.694 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:51.702+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:51.718 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:51.736 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:51.754 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:51.773 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.16571169599999, 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.775 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.165687281, 2024-04-24T18:09:51.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.781 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.16566413199999, 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.786 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:51.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.793 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.165583564, 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:51.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:51.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:51.800 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.16555875399999, 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:51.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.805 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.165538642, 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:51.807 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.809 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:51.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.813 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.165467979, 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.814 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:51.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:51.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 151.16545620599999, 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.820 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:51.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:51.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:51.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:51.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:51.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:51.826 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 150.745134377, 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.827 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:51.832 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 150.745087195, 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:51.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 150.745062074, 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:51.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:51.843 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.844 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.844 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.844 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.844 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:51.844 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 150.74500993999999, 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.846 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:51.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:51.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:51.849 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 150.744946476, 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:51.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:51.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:51.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:51.854 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.855 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 118.971447759, 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.856 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.857 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.858 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.859 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 118.97034169, 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.860 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.861 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.862 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:51.863 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 79.815169972000007, 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.864 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:51.865 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:51.866 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.867 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.753216027999997, 2024-04-24T18:09:51.868 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.869 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.870 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.871 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.872 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.751928945000003, 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:51.873 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:51.874 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:51.875 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:51.876 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.412752873999999, 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:51.877 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.878 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:51.879 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:51.880 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 51.412663180000003, 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:51.881 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.882 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:51.883 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:51.884 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:51.885 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:51.886 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:51.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:51.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:51.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:51.887 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:51.888 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:51.889 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:51.889 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:51.889 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:51.889 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:51.889 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:51.889 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:51.889 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:51.889 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:51.895+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:51.987 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:51.996+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:52.013 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:52.023 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:52.089 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:52.097+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:52.146 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:52.146 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:52.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:52.190 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:52.198+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:52.279 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:52.280 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:52.289 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:52.290 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:52.298+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:52.391 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:52.400+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:52.425 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:52.426 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:52.427 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:09:52.440 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:52.524 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:52.532+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:52.594 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:52.602+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:52.618 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:52.619 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:52.629 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:52.713 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:52.721+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:52.795 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:52.803+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:52.798 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:52.798 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:52.808 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:52.897 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:52.905+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:52.977 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:52.977 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:52.990 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:52.998 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:53.007+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:53.099 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:53.107+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:53.118 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:53.118 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:53.128 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:53.213 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:53.221+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:53.297 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:53.297 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:53.300 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:53.308+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:53.307 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:53.401 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:53.410+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:53.492 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:53.492 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:53.502 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:53.510+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:53.504 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:53.603 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:53.611+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:53.674 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:53.675 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:53.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:53.703 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:53.712+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:53.804 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:53.812+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:53.813 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:53.813 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:53.823 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:53.890 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:53.907 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:53.915+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:53.997 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:54.060 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:54.062 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:54.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:54.063 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:54.070+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:54.074 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:54.094 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:54.113 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:54.133 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:54.152 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:54.162 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:54.171+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:54.171 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:54.191 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:54.192 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:54.192 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:09:54.192 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:09:54.192 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:09:54.192 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:54.193 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:54.204 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:54.264 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:54.272+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:54.364 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:54.372+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:54.374 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.20311124600001, 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:09:54.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:09:54.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18347212, 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.379 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.382 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.183440398, 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.387 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18341698099999, 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.388 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.183386908, 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.392 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.182341761, 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.396 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18232453, 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.400 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:09:54.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:09:54.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.182307532, 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:09:54.404 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:54.408 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.182288851, 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:09:54.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.182271383, 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.413 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:09:54.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.182255259, 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:09:54.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.423 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18222415300001, 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.424 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:09:54.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18220495200001, 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:09:54.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.432 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18218391000001, 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:54.436 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.181428962, 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18140246, 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:09:54.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:09:54.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.18138492999999, 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:09:54.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.17992177000001, 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:09:54.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:09:54.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.168083122, 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:09:54.452 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:54.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:09:54.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 102.168066779, 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:09:54.456 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:09:54.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:09:54.460 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:09:54.460 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:09:54.462 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:54.504 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:54.511+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:54.565 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:54.573+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:54.594 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:54.594 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:54.595 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:54.595 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:54.604 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:54.667 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:54.675+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:54.728 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:54.728 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:54.728 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:54.729 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:54.729 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:54.738 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:54.779 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:54.788+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:54.869 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:54.877+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:54.904 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:54.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.572127792, 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:54.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:54.908 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.572111886, 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.909 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.572059919, 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.913 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:54.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.916 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.57113644499999, 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.917 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:54.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.571119813, 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.921 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:54.925 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.57041821, 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:54.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:54.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.57040170800001, 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:54.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.569498842, 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.934 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:54.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.937 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.569490817, 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.938 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:54.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:54.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:54.942 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.568426675, 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.946 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 125.56841070599999, 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.950 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 88.508212267000005, 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:54.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:54.956 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.339188234999995, 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:54.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.339172000999994, 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:54.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.339156781, 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:54.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:54.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.339129201999995, 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.339110900999998, 2024-04-24T18:09:54.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:54.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.977 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 78.339094602000003, 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.030691982999997, 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:54.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:54.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 55.681418878000002, 2024-04-24T18:09:54.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:54.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:54.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.991 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:54.991 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:54.991 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:54.992 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:54.993 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:55.001+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:55.094 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:55.102+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:55.157 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:55.157 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:55.157 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:55.157 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:55.167 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:55.210 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:55.219+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:55.290 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:55.290 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:55.290 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:55.291 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:55.291 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:55.295 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:55.303+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:55.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:55.396 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:55.404+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:55.449 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:09:55.471 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.471 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114924009999996, 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:55.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.475 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114915881000002, 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.476 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114907857999995, 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114898944999993, 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:55.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.488 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114890126000006, 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:55.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.492 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114880994000004, 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:55.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:55.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:55.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114060734999995, 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:55.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114033535999994, 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:55.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:55.505 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114016066000005, 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:55.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:55.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.114000726, 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:55.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:55.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113976648000005, 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.514 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:55.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113961083000007, 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.518 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:55.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:55.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:55.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113946122000002, 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:55.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:55.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:55.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.526 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113929182999996, 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:55.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113912778, 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.531 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:55.533 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:55.534 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113896338999993, 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.535 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.536 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.537 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.538 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113881499000001, 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.539 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.540 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:55.541 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:55.542 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113865075000007, 2024-04-24T18:09:55.543 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.544 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.545 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.546 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.547 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113847860000007, 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.548 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:55.549 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.550 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.551 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.113830394999994, 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:55.552 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:55.553 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.554 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.555 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.556 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.556 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:55.557 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:55.557 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:55.565+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:55.658 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:55.665+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:55.681 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:55.681 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:55.681 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:55.681 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:55.691 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:55.758 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:55.767+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:55.814 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:55.814 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:55.814 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:55.814 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:55.814 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:55.823 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:09:55.865 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:55.873+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:55.960 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:55.969+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:55.989 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:55.989 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.38043263099999, 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:55.990 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:09:55.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:09:55.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:09:55.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.38040821600001, 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:55.996 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:55.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:09:55.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:09:56.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.38038506699999, 2024-04-24T18:09:56.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:09:56.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:09:56.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.008 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.380304499, 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:09:56.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.38027968899999, 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.015 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:09:56.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:09:56.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.021 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.380259577, 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.022 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:56.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:09:56.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:09:56.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:09:56.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:56.028 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.380188914, 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:09:56.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.034 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 155.38017714099999, 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.035 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:56.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:09:56.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.959855312, 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.041 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:09:56.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:09:56.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.047 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.95980813, 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:09:56.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.050 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.051 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:09:56.052 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.053 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.95978300900001, 2024-04-24T18:09:56.054 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:56.055 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.056 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:09:56.057 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:09:56.058 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.059 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:09:56.060 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.95973087499999, 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.061 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.062 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:09:56.063 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:09:56.065 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.065 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.065 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.065 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:09:56.066 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:09:56.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:09:56.069 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.959667411, 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.070 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.071 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:09:56.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:09:56.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.075 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 123.186168694, 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:09:56.076 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:09:56.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 123.185062625, 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:09:56.081 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.083 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:09:56.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 84.029890906999995, 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.086 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:09:56.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:09:56.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 55.967936963, 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:09:56.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:09:56.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 55.966649879999999, 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:09:56.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:09:56.100 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 55.627473809000001, 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:09:56.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 55.627384114999998, 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:09:56.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:09:56.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:09:56.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:09:56.110 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.110 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.110 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.110 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:56.110 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:56.110 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:56.111 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:09:56.112 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:56.120+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:56.206 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:09:56.250 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:56.258+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:56.276 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:56.276 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:56.276 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:56.276 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:56.286 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:09:56.329 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:56.338+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:56.365 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:56.383 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:56.401 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:56.409 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:56.409 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:56.410 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:56.410 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:56.410 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:56.414 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:56.422+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:56.419 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:09:56.420 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:56.437 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:09:56.437 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:09:56.438 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-04-24T18:09:56.438 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 0.05 2024-04-24T18:09:56.438 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:56.466 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:56.491 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:56.515 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:56.523+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:56.516 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:56.517 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:56.518 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:56.518 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:56.518 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:56.518 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:56.518 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:56.518 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:56.518 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:56.594 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:56.594 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:56.604 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:09:56.648 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:56.656+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:56.718 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:56.726+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:56.734 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:56.734 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:56.744 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:09:56.795 INFO:teuthology.orchestra.run.smithi049.stdout:moved 0 / 90 (0%) 2024-04-24T18:09:56.795 INFO:teuthology.orchestra.run.smithi049.stdout:avg 11.25 2024-04-24T18:09:56.795 INFO:teuthology.orchestra.run.smithi049.stdout:stddev 3.07205 -> 3.07205 (expected baseline 3.13748) 2024-04-24T18:09:56.795 INFO:teuthology.orchestra.run.smithi049.stdout:min osd.2 with 6 -> 6 pgs (0.533333 -> 0.533333 * mean) 2024-04-24T18:09:56.795 INFO:teuthology.orchestra.run.smithi049.stdout:max osd.1 with 16 -> 16 pgs (1.42222 -> 1.42222 * mean) 2024-04-24T18:09:56.796 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:56.796 INFO:teuthology.orchestra.run.smithi049.stdout:oload 110 2024-04-24T18:09:56.796 INFO:teuthology.orchestra.run.smithi049.stdout:max_change 0.05 2024-04-24T18:09:56.796 INFO:teuthology.orchestra.run.smithi049.stdout:max_change_osds 4 2024-04-24T18:09:56.796 INFO:teuthology.orchestra.run.smithi049.stdout:average_utilization 0.0033 2024-04-24T18:09:56.796 INFO:teuthology.orchestra.run.smithi049.stdout:overload_utilization 0.0036 2024-04-24T18:09:56.798 INFO:teuthology.orchestra.run.smithi049.stderr:no change 2024-04-24T18:09:56.810 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-24T18:09:56.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 3.0 2024-04-24T18:09:56.851 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:56.859+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:56.914 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:56.914 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:56.919 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:56.927+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:56.925 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:09:57.021 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:57.029+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:57.096 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:57.097 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:57.107 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:09:57.152 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:57.160+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:57.188 INFO:teuthology.orchestra.run.smithi049.stdout:moved 0 / 90 (0%) 2024-04-24T18:09:57.188 INFO:teuthology.orchestra.run.smithi049.stdout:avg 11.25 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:stddev 3.07205 -> 3.07205 (expected baseline 3.13748) 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:min osd.2 with 6 -> 6 pgs (0.533333 -> 0.533333 * mean) 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:max osd.1 with 16 -> 16 pgs (1.42222 -> 1.42222 * mean) 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:oload 1000 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:max_change 3 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:max_change_osds 4 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:average_utilization 0.0033 2024-04-24T18:09:57.189 INFO:teuthology.orchestra.run.smithi049.stdout:overload_utilization 0.0327 2024-04-24T18:09:57.191 INFO:teuthology.orchestra.run.smithi049.stderr:no change 2024-04-24T18:09:57.203 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '3.0', 'overage': '110', 'type': 'reweight-by-utilization'} 2024-04-24T18:09:57.203 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 110 3.0 2024-04-24T18:09:57.224 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:57.232+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:57.240 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:57.240 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:57.252 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:09:57.325 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:57.333+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:57.426 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:57.434+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:57.450 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:57.450 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:57.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:09:57.468 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:09:57.526 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:57.535+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:57.591 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:57.591 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:57.602 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:09:57.611 INFO:teuthology.orchestra.run.smithi049.stdout:moved 0 / 90 (0%) 2024-04-24T18:09:57.611 INFO:teuthology.orchestra.run.smithi049.stdout:avg 11.25 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:stddev 3.07205 -> 3.07205 (expected baseline 3.13748) 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:min osd.2 with 6 -> 6 pgs (0.533333 -> 0.533333 * mean) 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:max osd.1 with 16 -> 16 pgs (1.42222 -> 1.42222 * mean) 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:oload 110 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:max_change 3 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:max_change_osds 4 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:average_utilization 0.0033 2024-04-24T18:09:57.612 INFO:teuthology.orchestra.run.smithi049.stdout:overload_utilization 0.0036 2024-04-24T18:09:57.615 INFO:teuthology.orchestra.run.smithi049.stderr:no change 2024-04-24T18:09:57.627 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '1000', 'type': 'reweight-by-utilization'} 2024-04-24T18:09:57.627 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight-by-utilization 1000 0.05 2024-04-24T18:09:57.628 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:57.635+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:57.728 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:57.736+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:57.748 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:57.748 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:57.759 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:09:57.829 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:57.837+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:57.929 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:57.930 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:57.930 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:57.938+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:57.940 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:moved 0 / 90 (0%) 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:avg 11.25 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:stddev 3.07205 -> 3.07205 (expected baseline 3.13748) 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:min osd.2 with 6 -> 6 pgs (0.533333 -> 0.533333 * mean) 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:max osd.1 with 16 -> 16 pgs (1.42222 -> 1.42222 * mean) 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:oload 1000 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:max_change 0.05 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:max_change_osds 4 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:average_utilization 0.0033 2024-04-24T18:09:58.008 INFO:teuthology.orchestra.run.smithi049.stdout:overload_utilization 0.0327 2024-04-24T18:09:58.011 INFO:teuthology.orchestra.run.smithi049.stderr:no change 2024-04-24T18:09:58.022 INFO:tasks.thrashosds.thrasher:Reweighting by: {'max_change': '0.05', 'overage': '110', 'type': 'test-reweight-by-utilization'} 2024-04-24T18:09:58.023 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd test-reweight-by-utilization 110 0.05 2024-04-24T18:09:58.031 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:58.039+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:58.070 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:58.070 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:58.081 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:09:58.131 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:58.140+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:58.212 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:58.212 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:58.223 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:09:58.267 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:58.275+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:58.324 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:09:58.333 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:58.341+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:58.370 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:58.370 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:moved 0 / 90 (0%) 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:avg 11.25 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:stddev 3.07205 -> 3.07205 (expected baseline 3.13748) 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:min osd.2 with 6 -> 6 pgs (0.533333 -> 0.533333 * mean) 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:max osd.1 with 16 -> 16 pgs (1.42222 -> 1.42222 * mean) 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:oload 110 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:max_change 0.05 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:max_change_osds 4 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:average_utilization 0.0033 2024-04-24T18:09:58.376 INFO:teuthology.orchestra.run.smithi049.stdout:overload_utilization 0.0036 2024-04-24T18:09:58.379 INFO:teuthology.orchestra.run.smithi049.stderr:no change 2024-04-24T18:09:58.384 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:09:58.435 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:58.443+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:58.534 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:09:58.535 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:58.543+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:58.566 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:58.566 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:58.577 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:09:58.636 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:58.644+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:58.679 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:09:58.697 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:09:58.715 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:09:58.716 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:09:58.716 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:09:58.727 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:09:58.735 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:09:58.737 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:58.745+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:58.753 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:09:58.771 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:09:58.790 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:09:58.809 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:09:58.810 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:09:58.810 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:09:58.810 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:09:58.811 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:09:58.811 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:09:58.811 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:09:58.811 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:09:58.811 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:09:58.838 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:09:58.846+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:09:58.892 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:58.892 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:58.892 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:58.892 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:58.892 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:58.901 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:09:58.939 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:58.948+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:59.040 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:59.048+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:59.066 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:59.066 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:59.066 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:59.066 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:59.066 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73448909199999, 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.067 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.068 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.069 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.070 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.734473186, 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.071 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:09:59.072 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:09:59.073 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:09:59.074 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73442121900001, 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.075 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.076 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:09:59.077 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:09:59.078 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73349774499999, 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.079 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.080 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.081 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.082 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73348111300001, 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.083 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.084 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.085 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.086 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73277951, 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.087 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.088 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:09:59.089 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:09:59.090 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73276300800001, 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.091 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.092 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:09:59.093 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:09:59.094 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73186014199999, 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:09:59.095 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.096 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:59.097 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.097 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.097 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.097 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.097 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.098 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.099 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.73185211699999, 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.100 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.101 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.102 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.103 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.730787975, 2024-04-24T18:09:59.104 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.105 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.106 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.107 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:59.108 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 129.730772006, 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.109 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.110 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.111 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:59.112 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 92.670573567000005, 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.113 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.114 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.115 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:09:59.116 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.117 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 82.501549534999995, 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.118 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.119 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.120 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.121 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 82.501533300999995, 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.122 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.123 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.124 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.125 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 82.501518081, 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.126 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:09:59.127 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:09:59.128 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.129 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 82.501490501999996, 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 82.501472200999999, 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.137 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:09:59.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:09:59.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:09:59.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 82.501455902000004, 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:09:59.141 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:09:59.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:09:59.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.193053282999998, 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.146 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:09:59.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:09:59.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:09:59.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 59.843780178000003, 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.150 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:09:59.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:09:59.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.154 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:59.156 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:09:59.156 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:09:59.164+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:09:59.256 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:09:59.264+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:09:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:59.324 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:59.324 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:59.324 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:59.335 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:09:59.356 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:09:59.364+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:09:59.457 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:09:59.466+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:09:59.467 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:59.467 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:59.467 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:09:59.467 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:09:59.468 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:59.478 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:09:59.561 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:09:59.570+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:09:59.643 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:59.643 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.287090673999998, 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.644 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.645 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.646 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:09:59.647 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:09:59.647 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.647 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.647 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.647 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:09:59.647 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:09:59.649 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.649 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.649 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:09:59.650 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.287082545000004, 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.651 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.652 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.653 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:09:59.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.655 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.287074521999998, 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.656 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:09:59.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.660 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.287065608999995, 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:09:59.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:09:59.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:59.665 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.287056789999994, 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:59.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:09:59.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.287047658000006, 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:09:59.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286227398999998, 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.675 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:09:59.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286200199999996, 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.680 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:09:59.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:09:59.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.685 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286182729999993, 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.686 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:09:59.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286167390000003, 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.691 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:09:59.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286143311999993, 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:09:59.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.698 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:09:59.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:59.700 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286127746999995, 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:59.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:09:59.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286112786000004, 2024-04-24T18:09:59.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:59.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:09:59.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286095846999999, 2024-04-24T18:09:59.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:59.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:09:59.715 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286079442000002, 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.716 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:09:59.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.720 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286063002999995, 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:09:59.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:09:59.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286048163000004, 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.726 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:09:59.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.730 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286031738999995, 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:09:59.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.286014523999995, 2024-04-24T18:09:59.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.737 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:09:59.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 89.285997058999996, 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:09:59.740 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:09:59.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:09:59.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:09:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:09:59.745 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:09:59.745 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:59.747 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:09:59.747 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:59.755+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:09:59.848 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:09:59.856+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:09:59.911 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:09:59.911 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:09:59.911 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:09:59.911 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:09:59.920 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:09:59.949 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:09:59.957+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:00.043 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:00.044 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:00.044 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:00.044 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:00.044 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:00.050 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:00.059+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:00.053 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:10:00.152 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:00.160+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:00.180 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.571539194, 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.181 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:10:00.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:10:00.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:10:00.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.57151477900001, 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:10:00.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:10:00.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:10:00.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.191 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.57149163, 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.192 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:10:00.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:10:00.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.57141106200001, 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.197 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:10:00.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:00.202 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.571386252, 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.207 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.57136614000001, 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:10:00.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:10:00.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.57129547700001, 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.213 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:10:00.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.218 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.571283704, 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:10:00.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:10:00.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.15096187500001, 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.224 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:10:00.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:00.229 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.150914693, 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.231 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:10:00.232 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:10:00.233 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:10:00.234 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.15088957200001, 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.235 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.236 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.237 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.238 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.239 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:00.240 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.150837438, 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.241 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.242 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:10:00.243 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.244 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.245 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 159.150773974, 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.246 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.247 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.248 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:00.249 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.250 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 127.37727525699999, 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.251 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.252 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.253 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.254 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 127.37616918800001, 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.255 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.256 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.257 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.258 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 88.22099747, 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.259 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:00.260 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.261 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.262 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 60.159043525999998, 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.263 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.264 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.265 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.266 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 60.157756442999997, 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.267 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.268 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.269 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.270 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 59.818580372, 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.271 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:10:00.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:10:00.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:10:00.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 59.818490678000003, 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:10:00.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:10:00.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:10:00.279 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.279 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.279 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.279 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.279 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.279 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:00.280 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:10:00.281 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:00.289+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:00.342 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:10:00.381 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:00.389+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:00.415 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:00.415 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:00.415 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:00.415 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:00.426 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:10:00.510 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:00.519+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:00.583 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:00.591+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:00.592 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:00.592 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:00.592 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:00.592 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:00.592 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:00.602 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:10:00.686 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:00.695+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:00.768 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.257198141000003, 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:10:00.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.771 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.257177206000001, 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.773 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:10:00.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:10:00.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.257147447999998, 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.780 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:00.781 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:10:00.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253642954, 2024-04-24T18:10:00.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:10:00.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:10:00.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.788 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253618619999997, 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:00.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253597030000002, 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253580904000003, 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:10:00.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.800 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253566010999997, 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.801 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:10:00.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253311244999999, 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:10:00.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:10:00.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:10:00.808 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253283449999998, 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.809 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:10:00.810 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:10:00.811 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:10:00.812 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253251185000003, 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.813 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.814 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.815 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.816 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.253218261000001, 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.817 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.818 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.819 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.820 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251303161999999, 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.821 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.822 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.823 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.824 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:00.825 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251293154000003, 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.826 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:10:00.827 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:10:00.828 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.829 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251279707000002, 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.830 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.831 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.832 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.833 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251271074000002, 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.834 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.835 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.836 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.837 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251262576999999, 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.838 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:10:00.839 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:10:00.840 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.841 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251254940999999, 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.842 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.843 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:10:00.844 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.845 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251247286999998, 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:10:00.846 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:00.847 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:10:00.848 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.849 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 61.251230737, 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:10:00.850 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:00.851 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.852 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.853 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.854 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:00.854 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:00.854 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:00.854 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:10:00.856 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:00.863+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:00.856 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:10:00.955 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:00.963+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:00.992 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:00.992 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:00.992 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:00.992 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:01.006 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:10:01.013 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:01.031 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:01.051 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:01.056 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:01.065+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:01.070 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:01.089 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:01.108 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:01.127 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:01.144 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:01.144 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:01.145 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:01.147 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:01.156 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:10:01.157 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:01.165+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:01.258 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:01.267+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:01.326 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:01.326 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:01.336 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:10:01.380 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:01.388+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:01.459 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:01.467+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:01.460 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:10:01.470 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:01.470 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:01.482 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:10:01.560 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:01.569+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:01.658 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:01.658 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:01.661 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:01.670+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:01.668 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:10:01.763 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:01.772+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:01.796 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:01.797 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:01.807 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:10:01.864 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:01.872+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:01.964 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:01.973+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:01.976 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:01.976 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:01.987 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:10:02.065 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:02.074+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:02.115 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:02.115 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:02.125 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:10:02.169 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:02.178+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:02.254 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:02.254 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:02.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:10:02.266 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:02.275+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:02.367 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:02.376+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:02.456 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:02.457 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:02.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:10:02.468 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:02.477+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:02.569 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:02.577+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:02.596 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:02.596 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:02.606 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:10:02.669 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:02.678+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:02.734 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:02.734 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:02.744 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:10:02.789 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:02.798+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:02.871 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:02.880+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:02.882 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:02.882 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:02.892 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:10:02.976 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:02.985+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:03.057 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:03.057 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:03.067 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:10:03.109 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:03.117+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:03.159 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:10:03.203 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:03.212+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:03.231 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:03.231 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:03.241 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:10:03.274 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:03.282+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:03.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:03.344 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:03.368 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:03.374 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:03.383+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:03.375 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:03.375 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:03.387 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:10:03.391 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:03.393 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:10:03.393 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:10:03.394 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-24T18:10:03.394 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-24T18:10:03.412 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:03.435 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:03.454 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:03.474 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:10:03.475 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:03.475 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:03.484+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:03.476 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:03.516 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:03.516 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:03.516 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:03.517 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:03.517 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:03.526 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:10:03.576 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:03.585+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:03.653 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:03.653 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:03.653 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.297177165999997, 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:10:03.654 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:10:03.655 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.656 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:10:03.657 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.297169037000003, 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.658 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:03.659 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:10:03.660 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:10:03.661 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.297161013999997, 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.662 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.663 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:10:03.664 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.665 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.297152100999995, 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.666 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.667 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.668 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.669 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.297143281999993, 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.670 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.671 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.672 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.673 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.297134150000005, 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.674 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.675 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.676 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.677 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296313890999997, 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.678 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.679 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.680 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.681 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296286691999995, 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.682 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.683 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.684 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:10:03.685 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296269222000006, 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.686 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:10:03.687 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:10:03.688 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.689 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296253882000002, 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:10:03.690 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:03.691 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:10:03.692 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:10:03.693 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296229804000006, 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:10:03.694 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.695 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:10:03.696 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:10:03.697 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296214238999994, 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.698 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.699 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.700 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.701 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296199278000003, 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.702 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.703 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.704 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.705 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296182338999998, 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.706 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.707 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.708 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.709 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.710 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296165934000001, 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.711 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.712 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.713 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.714 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296149494999995, 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:03.715 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.716 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.717 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.718 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296134655000003, 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.719 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.720 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:10:03.721 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.722 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296118230999994, 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.723 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296101015999994, 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.730 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 93.296083550999995, 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:03.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:10:03.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.734 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.735 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:03.735 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:03.735 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:03.736 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:10:03.737 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:03.745+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:03.740 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:03.740 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:03.740 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 155, 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "fsid": "5dae32d6-9101-4f7b-a6ef-ca5186fee71b", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "created": "2024-04-24T18:07:07.515859+0000", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "modified": "2024-04-24T18:10:03.311309+0000", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "last_up_change": "2024-04-24T18:08:52.865354+0000", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "last_in_change": "2024-04-24T18:07:55.610486+0000", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_num": 5801984, 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_set": [ 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "noscrub", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "pglog_hardlimit", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "purged_snapdirs", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_deletes", 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "sortbitwise" 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.741 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_version": 5, 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "full_ratio": 0.94999998807907104, 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_snapshot": "", 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_max": 4, 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd": 8, 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "require_min_compat_client": "luminous", 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "min_compat_client": "jewel", 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "require_osd_release": "quincy", 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "pools": [ 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 1, 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "rbd", 2024-04-24T18:10:03.742 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:18.850062+0000", 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 8, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 8, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 8, 2024-04-24T18:10:03.743 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 8, 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 8, 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "21", 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "21", 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:03.744 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 19, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [ 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: 3 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": 3, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": 3, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:03.745 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:03.746 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 2, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": ".mgr", 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:19.353026+0000", 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-04-24T18:10:03.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 1, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 1, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 1, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 1, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 1, 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:03.748 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "17", 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:03.749 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:03.750 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:03.751 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_max": 32, 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 1 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": {} 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 3, 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "cache", 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:25.800822+0000", 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 9, 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool,incomplete_clones", 2024-04-24T18:10:03.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 4, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 4, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 4, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 4, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 4, 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:03.753 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "25", 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "21", 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 19, 2024-04-24T18:10:03.754 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": 1, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "writeback", 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 250, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:03.755 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bloom", 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "false_positive_probability": 0.050000000000000003, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "target_size": 0, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "seed": 0 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 60, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 8, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:03.756 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [ 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:03.757 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 4, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "pool_client.0", 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:35.513947+0000", 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 8193, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:03.758 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 32, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 32, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 32, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 32, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 32, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:03.759 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "155", 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 98, 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 155, 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:10:03.760 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:10:03.761 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.762 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "osds": [ 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "24da2d2a-953e-4440-9f6d-fb19cbec9ca1", 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 69, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6803", 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6805", 2024-04-24T18:10:03.764 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6807", 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6813", 2024-04-24T18:10:03.765 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6815", 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6809", 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.766 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6811", 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6803/43474", 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6807/43474", 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6815/43474", 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6811/43474", 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.767 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "a47a3673-abbd-4827-966a-d68b17f3a18f", 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 0.854644775390625, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 98, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:03.768 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6802", 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.769 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6804", 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6806", 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.770 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6812", 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6814", 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.771 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6808", 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6810", 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6802/43473", 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6806/43473", 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6814/43473", 2024-04-24T18:10:03.772 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6810/43473", 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "de3910ae-cc81-45d4-8639-cf2415330e64", 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:03.773 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:03.774 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 45, 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6824", 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6825", 2024-04-24T18:10:03.775 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6826", 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6827", 2024-04-24T18:10:03.776 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6830", 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6831", 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.777 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6828", 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6829", 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.778 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6825/43481", 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6827/43481", 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6831/43481", 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6829/43481", 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0", 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.779 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 69, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6816", 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.780 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6817", 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6818", 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.781 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6819", 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6822", 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.782 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6823", 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6820", 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.783 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6821", 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6817/43475", 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6819/43475", 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6823/43475", 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6821/43475", 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.784 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "47f1fd84-e601-4e48-8058-bfa9296a9f90", 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 99, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.785 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6800", 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6801", 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6802", 2024-04-24T18:10:03.786 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6803", 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6806", 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.787 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6807", 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6804", 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.788 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6805", 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6801/43499", 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6803/43499", 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6807/43499", 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6805/43499", 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.789 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "1796d83b-591c-4c9d-b376-e36a2416a17e", 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 97, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.790 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6808", 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6809", 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.791 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6810", 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6811", 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.792 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6814", 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6815", 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.793 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6812", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6813", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6809/43500", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6811/43500", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6815/43500", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6813/43500", 2024-04-24T18:10:03.794 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "5f203bac-3db4-4129-96cc-63dc06ee5ffa", 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 13, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 67, 2024-04-24T18:10:03.795 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 95, 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 98, 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 69, 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6824", 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6825", 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.796 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6826", 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6827", 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.797 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6830", 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6831", 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.798 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6828", 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6829", 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.799 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6825/58122", 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6827/58122", 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6831/58122", 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6829/58122", 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "51061805-fbe5-4e80-be6b-dfc37cec0221", 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:03.800 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 48, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6816", 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.801 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6817", 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6818", 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.802 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6819", 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6822", 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.803 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6823", 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6820", 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:03.804 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6821", 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6817/43501", 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6819/43501", 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6823/43501", 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6821/43501", 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.805 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_xinfo": [ 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.248445+0000", 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.806 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.246633+0000", 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.176542+0000", 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.807 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.192111+0000", 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.808 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.151845+0000", 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.809 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.187089+0000", 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.255937+0000", 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 69 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.810 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.160381+0000", 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap": [], 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap_items": [], 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_temp": [], 2024-04-24T18:10:03.811 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_temp": [], 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklist": {}, 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "range_blocklist": {}, 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profiles": { 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "default": { 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-failure-domain": "osd", 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "k": "2", 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "m": "1", 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin": "jerasure", 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "technique": "reed_sol_van" 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps_queue": [ 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 4, 2024-04-24T18:10:03.812 INFO:teuthology.orchestra.run.smithi049.stdout: "snaps": [ 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: "begin": 13, 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: "length": 7 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: "new_removed_snaps": [ 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 4, 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: "snaps": [ 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: "begin": 13, 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: "length": 1 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.813 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "new_purged_snaps": [], 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_node_flags": {}, 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "device_class_flags": {}, 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode": { 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_enabled": false, 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_bucket_count": 0, 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "degraded_stretch_mode": 0, 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "recovering_stretch_mode": 0, 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_bucket": 0 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:03.814 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:03.816 INFO:tasks.thrashosds.thrasher:j is {'epoch': 155, 'fsid': '5dae32d6-9101-4f7b-a6ef-ca5186fee71b', 'created': '2024-04-24T18:07:07.515859+0000', 'modified': '2024-04-24T18:10:03.311309+0000', 'last_up_change': '2024-04-24T18:08:52.865354+0000', 'last_in_change': '2024-04-24T18:07:55.610486+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-24T18:07:18.850062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-04-24T18:07:19.353026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-04-24T18:07:25.800822+0000', 'flags': 9, 'flags_names': 'hashpspool,incomplete_clones', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-04-24T18:07:35.513947+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '155', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 98, 'snap_epoch': 155, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '24da2d2a-953e-4440-9f6d-fb19cbec9ca1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6801', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6803', 'nonce': 43474}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6805', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6807', 'nonce': 43474}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6813', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6815', 'nonce': 43474}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6809', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6811', 'nonce': 43474}]}, 'public_addr': '172.21.15.49:6803/43474', 'cluster_addr': '172.21.15.49:6807/43474', 'heartbeat_back_addr': '172.21.15.49:6815/43474', 'heartbeat_front_addr': '172.21.15.49:6811/43474', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a47a3673-abbd-4827-966a-d68b17f3a18f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.854644775390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 98, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6800', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6802', 'nonce': 43473}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6804', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6806', 'nonce': 43473}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6812', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6814', 'nonce': 43473}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6808', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6810', 'nonce': 43473}]}, 'public_addr': '172.21.15.49:6802/43473', 'cluster_addr': '172.21.15.49:6806/43473', 'heartbeat_back_addr': '172.21.15.49:6814/43473', 'heartbeat_front_addr': '172.21.15.49:6810/43473', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'de3910ae-cc81-45d4-8639-cf2415330e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 45, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6824', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6825', 'nonce': 43481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6826', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6827', 'nonce': 43481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6830', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6831', 'nonce': 43481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6828', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6829', 'nonce': 43481}]}, 'public_addr': '172.21.15.49:6825/43481', 'cluster_addr': '172.21.15.49:6827/43481', 'heartbeat_back_addr': '172.21.15.49:6831/43481', 'heartbeat_front_addr': '172.21.15.49:6829/43481', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 69, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6816', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6817', 'nonce': 43475}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6818', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6819', 'nonce': 43475}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6822', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6823', 'nonce': 43475}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6820', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6821', 'nonce': 43475}]}, 'public_addr': '172.21.15.49:6817/43475', 'cluster_addr': '172.21.15.49:6819/43475', 'heartbeat_back_addr': '172.21.15.49:6823/43475', 'heartbeat_front_addr': '172.21.15.49:6821/43475', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '47f1fd84-e601-4e48-8058-bfa9296a9f90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 99, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6800', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6801', 'nonce': 43499}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6802', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6803', 'nonce': 43499}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6806', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6807', 'nonce': 43499}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6804', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6805', 'nonce': 43499}]}, 'public_addr': '172.21.15.78:6801/43499', 'cluster_addr': '172.21.15.78:6803/43499', 'heartbeat_back_addr': '172.21.15.78:6807/43499', 'heartbeat_front_addr': '172.21.15.78:6805/43499', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1796d83b-591c-4c9d-b376-e36a2416a17e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 97, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6808', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6809', 'nonce': 43500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6810', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6811', 'nonce': 43500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6814', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6815', 'nonce': 43500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6812', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6813', 'nonce': 43500}]}, 'public_addr': '172.21.15.78:6809/43500', 'cluster_addr': '172.21.15.78:6811/43500', 'heartbeat_back_addr': '172.21.15.78:6815/43500', 'heartbeat_front_addr': '172.21.15.78:6813/43500', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5f203bac-3db4-4129-96cc-63dc06ee5ffa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 67, 'up_from': 95, 'up_thru': 98, 'down_at': 69, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6824', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6825', 'nonce': 58122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6826', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6827', 'nonce': 58122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6830', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6831', 'nonce': 58122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6828', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6829', 'nonce': 58122}]}, 'public_addr': '172.21.15.78:6825/58122', 'cluster_addr': '172.21.15.78:6827/58122', 'heartbeat_back_addr': '172.21.15.78:6831/58122', 'heartbeat_front_addr': '172.21.15.78:6829/58122', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '51061805-fbe5-4e80-be6b-dfc37cec0221', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 48, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6816', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6817', 'nonce': 43501}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6818', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6819', 'nonce': 43501}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6822', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6823', 'nonce': 43501}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6820', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6821', 'nonce': 43501}]}, 'public_addr': '172.21.15.78:6817/43501', 'cluster_addr': '172.21.15.78:6819/43501', 'heartbeat_back_addr': '172.21.15.78:6823/43501', 'heartbeat_front_addr': '172.21.15.78:6821/43501', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.248445+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.246633+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.176542+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.192111+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.151845+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.187089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.255937+0000', 'dead_epoch': 69}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.160381+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 4, 'snaps': [{'begin': 13, 'length': 7}]}], 'new_removed_snaps': [{'pool': 4, 'snaps': [{'begin': 13, 'length': 1}]}], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-24T18:10:03.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:10:03.857 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:03.866+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:03.860 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:03.860 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:03.860 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:03.860 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:03.870 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:10:03.937 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:03.946+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:03.993 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:03.994 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:03.994 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:03.994 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:03.994 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:04.004 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:10:04.038 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:04.047+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.51958265799999, 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.129 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.130 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:10:04.131 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.132 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.133 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.134 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.51955824300001, 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.135 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.136 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.137 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.138 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.139 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.51953509399999, 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.140 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.141 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.142 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:10:04.143 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:10:04.144 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:10:04.145 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.519454526, 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.146 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:10:04.147 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:10:04.148 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:10:04.149 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.150 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.151 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.51942971599999, 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.152 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.153 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.154 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.155 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.156 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.519409604, 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.157 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.158 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:10:04.159 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:10:04.160 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.161 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.162 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.519338941, 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.163 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:10:04.164 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.165 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.166 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.167 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.51932716799999, 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:10:04.168 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.169 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:10:04.170 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:10:04.171 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:10:04.172 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.173 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.099005339, 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:04.174 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:10:04.175 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.176 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.177 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.178 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.098958157, 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.179 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.180 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.181 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:10:04.182 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:10:04.183 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:10:04.184 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.09893303600001, 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.185 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.186 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:10:04.187 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:10:04.188 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:10:04.189 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.09888090199999, 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:10:04.190 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.191 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.192 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.193 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.194 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.195 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 163.098817438, 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.196 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.197 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.198 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.199 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.200 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 131.325318721, 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.201 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.202 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.203 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.204 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 131.324212652, 2024-04-24T18:10:04.205 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.206 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:10:04.207 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:10:04.208 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.209 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 92.169040933999995, 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:04.210 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:10:04.211 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:10:04.212 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.213 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.107086989999999, 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.214 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:10:04.215 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:10:04.216 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:10:04.217 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 64.105799907000005, 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:04.218 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:10:04.219 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:10:04.220 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:10:04.221 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 63.766623836000001, 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:10:04.222 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.223 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.224 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.225 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 63.766534141999998, 2024-04-24T18:10:04.226 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.227 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.228 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:04.229 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.230 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:04.233 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:10:04.234 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:04.240+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:04.234 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:04.236 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:10:04.236 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":231,"stamp":"2024-04-24T18:10:03.328735+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":295605414,"num_objects":2264,"num_object_clones":167,"num_object_copies":4528,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2259,"num_whiteouts":1936,"num_read":290487,"num_read_kb":859597,"num_write":28754,"num_write_kb":534019,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1031,"num_bytes_recovered":44332122,"num_keys_recovered":150,"num_objects_omap":14,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":25506,"ondisk_log_size":25506,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2466984,"kb_used_data":33552,"kb_used_omap":43,"kb_used_meta":2433236,"kb_avail":752507736,"statfs":{"total":773094113280,"available":770567921664,"internally_reserved":0,"allocated":34357248,"data_stored":153132755,"data_compressed":3830863,"data_compressed_allocated":10489856,"data_compressed_original":138176998,"omap_allocated":44665,"internal_metadata":2491634055},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-133387232,"num_objects":-145,"num_object_clones":-55,"num_object_copies":-290,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-145,"num_whiteouts":-35,"num_read":1340,"num_read_kb":1145,"num_write":227,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000777"},"pg_stats":[{"pgid":"4.1f","version":"152'634","reported_seq":6611,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.467811+0000","last_change":"2024-04-24T18:10:01.468068+0000","last_active":"2024-04-24T18:10:01.467811+0000","last_peered":"2024-04-24T18:10:01.467811+0000","last_clean":"2024-04-24T18:10:01.467811+0000","last_became_active":"2024-04-24T18:08:56.900267+0000","last_became_peered":"2024-04-24T18:08:56.900267+0000","last_unstale":"2024-04-24T18:10:01.467811+0000","last_undegraded":"2024-04-24T18:10:01.467811+0000","last_fullsized":"2024-04-24T18:10:01.467811+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T06:00:07.953531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001343604,"stat_sum":{"num_bytes":9125986,"num_objects":89,"num_object_clones":5,"num_object_copies":178,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":89,"num_whiteouts":77,"num_read":7042,"num_read_kb":29087,"num_write":586,"num_write_kb":11953,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":118,"num_bytes_recovered":6111929,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"f","length":"a"}]},{"pgid":"4.1e","version":"152'1300","reported_seq":7135,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.313469+0000","last_change":"2024-04-24T18:10:02.313965+0000","last_active":"2024-04-24T18:10:02.313469+0000","last_peered":"2024-04-24T18:10:02.313469+0000","last_clean":"2024-04-24T18:10:02.313469+0000","last_became_active":"2024-04-24T18:07:36.545309+0000","last_became_peered":"2024-04-24T18:07:36.545309+0000","last_unstale":"2024-04-24T18:10:02.313469+0000","last_undegraded":"2024-04-24T18:10:02.313469+0000","last_fullsized":"2024-04-24T18:10:02.313469+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1300,"log_dups_size":0,"ondisk_log_size":1300,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:46:55.892624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00081158900000000004,"stat_sum":{"num_bytes":3969811,"num_objects":88,"num_object_clones":7,"num_object_copies":176,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":88,"num_whiteouts":76,"num_read":10280,"num_read_kb":26069,"num_write":1857,"num_write_kb":10821,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.1d","version":"150'697","reported_seq":5904,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.323595+0000","last_change":"2024-04-24T18:10:02.323870+0000","last_active":"2024-04-24T18:10:02.323595+0000","last_peered":"2024-04-24T18:10:02.323595+0000","last_clean":"2024-04-24T18:10:02.323595+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:10:02.323595+0000","last_undegraded":"2024-04-24T18:10:02.323595+0000","last_fullsized":"2024-04-24T18:10:02.323595+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":697,"log_dups_size":0,"ondisk_log_size":697,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0061854839999999998,"stat_sum":{"num_bytes":2711272,"num_objects":78,"num_object_clones":5,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":70,"num_read":6263,"num_read_kb":18013,"num_write":620,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.1c","version":"154'1058","reported_seq":10125,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.352489+0000","last_change":"2024-04-24T18:10:02.328219+0000","last_active":"2024-04-24T18:10:02.352489+0000","last_peered":"2024-04-24T18:10:02.352489+0000","last_clean":"2024-04-24T18:10:02.352489+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:02.352489+0000","last_undegraded":"2024-04-24T18:10:02.352489+0000","last_fullsized":"2024-04-24T18:10:02.352489+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1058,"log_dups_size":0,"ondisk_log_size":1058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0044179420000000002,"stat_sum":{"num_bytes":15260322,"num_objects":58,"num_object_clones":4,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":50,"num_read":12139,"num_read_kb":23555,"num_write":1369,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.1b","version":"152'512","reported_seq":11551,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.325402+0000","last_change":"2024-04-24T18:10:02.325804+0000","last_active":"2024-04-24T18:10:02.325402+0000","last_peered":"2024-04-24T18:10:02.325402+0000","last_clean":"2024-04-24T18:10:02.325402+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:02.325402+0000","last_undegraded":"2024-04-24T18:10:02.325402+0000","last_fullsized":"2024-04-24T18:10:02.325402+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":512,"log_dups_size":0,"ondisk_log_size":512,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011093229999999999,"stat_sum":{"num_bytes":13022756,"num_objects":65,"num_object_clones":8,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":50,"num_read":10925,"num_read_kb":37139,"num_write":476,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.1a","version":"154'862","reported_seq":12172,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.355906+0000","last_change":"2024-04-24T18:10:02.337283+0000","last_active":"2024-04-24T18:10:02.355906+0000","last_peered":"2024-04-24T18:10:02.355906+0000","last_clean":"2024-04-24T18:10:02.355906+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:02.355906+0000","last_undegraded":"2024-04-24T18:10:02.355906+0000","last_fullsized":"2024-04-24T18:10:02.355906+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.011403118,"stat_sum":{"num_bytes":10776904,"num_objects":63,"num_object_clones":7,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":51,"num_read":14173,"num_read_kb":30416,"num_write":981,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.19","version":"154'717","reported_seq":9644,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.323226+0000","last_change":"2024-04-24T18:10:02.323531+0000","last_active":"2024-04-24T18:10:02.323226+0000","last_peered":"2024-04-24T18:10:02.323226+0000","last_clean":"2024-04-24T18:10:02.323226+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:02.323226+0000","last_undegraded":"2024-04-24T18:10:02.323226+0000","last_fullsized":"2024-04-24T18:10:02.323226+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":717,"log_dups_size":0,"ondisk_log_size":717,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0058459640000000004,"stat_sum":{"num_bytes":11977649,"num_objects":71,"num_object_clones":2,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":64,"num_read":10221,"num_read_kb":31214,"num_write":803,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.18","version":"154'596","reported_seq":11561,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.329176+0000","last_change":"2024-04-24T18:10:02.329574+0000","last_active":"2024-04-24T18:10:02.329176+0000","last_peered":"2024-04-24T18:10:02.329176+0000","last_clean":"2024-04-24T18:10:02.329176+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:10:02.329176+0000","last_undegraded":"2024-04-24T18:10:02.329176+0000","last_fullsized":"2024-04-24T18:10:02.329176+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0051037210000000003,"stat_sum":{"num_bytes":9761695,"num_objects":66,"num_object_clones":7,"num_object_copies":132,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":66,"num_whiteouts":53,"num_read":12333,"num_read_kb":36289,"num_write":582,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.17","version":"154'1344","reported_seq":6870,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.320699+0000","last_change":"2024-04-24T18:10:02.320699+0000","last_active":"2024-04-24T18:10:02.320699+0000","last_peered":"2024-04-24T18:10:02.320699+0000","last_clean":"2024-04-24T18:10:02.320699+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:10:02.320699+0000","last_undegraded":"2024-04-24T18:10:02.320699+0000","last_fullsized":"2024-04-24T18:10:02.320699+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1344,"log_dups_size":0,"ondisk_log_size":1344,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":1,"snaptrim_duration":0.0033450049999999999,"stat_sum":{"num_bytes":4474239,"num_objects":68,"num_object_clones":7,"num_object_copies":136,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":68,"num_whiteouts":57,"num_read":10550,"num_read_kb":29370,"num_write":2067,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.2","version":"150'1224","reported_seq":4873,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.314778+0000","last_change":"2024-04-24T18:10:02.315101+0000","last_active":"2024-04-24T18:10:02.314778+0000","last_peered":"2024-04-24T18:10:02.314778+0000","last_clean":"2024-04-24T18:10:02.314778+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:10:02.314778+0000","last_undegraded":"2024-04-24T18:10:02.314778+0000","last_fullsized":"2024-04-24T18:10:02.314778+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1224,"log_dups_size":0,"ondisk_log_size":1224,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001242768,"stat_sum":{"num_bytes":10671128,"num_objects":69,"num_object_clones":4,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":60,"num_read":8641,"num_read_kb":24164,"num_write":1706,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"1.7","version":"0'0","reported_seq":119,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-24T18:10:00.343142+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:10:00.343142+0000","last_peered":"2024-04-24T18:10:00.343142+0000","last_clean":"2024-04-24T18:10:00.343142+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:10:00.343142+0000","last_undegraded":"2024-04-24T18:10:00.343142+0000","last_fullsized":"2024-04-24T18:10:00.343142+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":119,"reported_epoch":152,"state":"active+clean","last_fresh":"2024-04-24T18:10:00.343391+0000","last_change":"2024-04-24T18:08:53.875049+0000","last_active":"2024-04-24T18:10:00.343391+0000","last_peered":"2024-04-24T18:10:00.343391+0000","last_clean":"2024-04-24T18:10:00.343391+0000","last_became_active":"2024-04-24T18:08:53.874300+0000","last_became_peered":"2024-04-24T18:08:53.874300+0000","last_unstale":"2024-04-24T18:10:00.343391+0000","last_undegraded":"2024-04-24T18:10:00.343391+0000","last_fullsized":"2024-04-24T18:10:00.343391+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:34:06.445650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"154'892","reported_seq":6585,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.373552+0000","last_change":"2024-04-24T18:10:02.316132+0000","last_active":"2024-04-24T18:10:02.373552+0000","last_peered":"2024-04-24T18:10:02.373552+0000","last_clean":"2024-04-24T18:10:02.373552+0000","last_became_active":"2024-04-24T18:07:36.540171+0000","last_became_peered":"2024-04-24T18:07:36.540171+0000","last_unstale":"2024-04-24T18:10:02.373552+0000","last_undegraded":"2024-04-24T18:10:02.373552+0000","last_fullsized":"2024-04-24T18:10:02.373552+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":892,"log_dups_size":0,"ondisk_log_size":892,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:48.324276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002644216,"stat_sum":{"num_bytes":20287164,"num_objects":80,"num_object_clones":9,"num_object_copies":160,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":80,"num_whiteouts":64,"num_read":7584,"num_read_kb":32218,"num_write":889,"num_write_kb":18809,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"3.3","version":"0'0","reported_seq":143,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.469245+0000","last_change":"2024-04-24T18:07:27.503645+0000","last_active":"2024-04-24T18:10:01.469245+0000","last_peered":"2024-04-24T18:10:01.469245+0000","last_clean":"2024-04-24T18:10:01.469245+0000","last_became_active":"2024-04-24T18:07:27.503354+0000","last_became_peered":"2024-04-24T18:07:27.503354+0000","last_unstale":"2024-04-24T18:10:01.469245+0000","last_undegraded":"2024-04-24T18:10:01.469245+0000","last_fullsized":"2024-04-24T18:10:01.469245+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:00:23.772232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":146,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.474773+0000","last_change":"2024-04-24T18:07:20.402703+0000","last_active":"2024-04-24T18:10:01.474773+0000","last_peered":"2024-04-24T18:10:01.474773+0000","last_clean":"2024-04-24T18:10:01.474773+0000","last_became_active":"2024-04-24T18:07:20.402407+0000","last_became_peered":"2024-04-24T18:07:20.402407+0000","last_unstale":"2024-04-24T18:10:01.474773+0000","last_undegraded":"2024-04-24T18:10:01.474773+0000","last_fullsized":"2024-04-24T18:10:01.474773+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:52:57.194372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"148'1027","reported_seq":6660,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.315513+0000","last_change":"2024-04-24T18:10:02.316644+0000","last_active":"2024-04-24T18:10:02.315513+0000","last_peered":"2024-04-24T18:10:02.315513+0000","last_clean":"2024-04-24T18:10:02.315513+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:02.315513+0000","last_undegraded":"2024-04-24T18:10:02.315513+0000","last_fullsized":"2024-04-24T18:10:02.315513+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1027,"log_dups_size":0,"ondisk_log_size":1027,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022986970000000002,"stat_sum":{"num_bytes":2632533,"num_objects":76,"num_object_clones":5,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":69,"num_read":8999,"num_read_kb":14434,"num_write":1253,"num_write_kb":16379,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"3.2","version":"106'3","reported_seq":146,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.521102+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:10:01.521102+0000","last_peered":"2024-04-24T18:10:01.521102+0000","last_clean":"2024-04-24T18:10:01.521102+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:01.521102+0000","last_undegraded":"2024-04-24T18:10:01.521102+0000","last_fullsized":"2024-04-24T18:10:01.521102+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":85,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":85,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":145,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.524391+0000","last_change":"2024-04-24T18:07:20.402305+0000","last_active":"2024-04-24T18:10:01.524391+0000","last_peered":"2024-04-24T18:10:01.524391+0000","last_clean":"2024-04-24T18:10:01.524391+0000","last_became_active":"2024-04-24T18:07:20.401990+0000","last_became_peered":"2024-04-24T18:07:20.401990+0000","last_unstale":"2024-04-24T18:10:01.524391+0000","last_undegraded":"2024-04-24T18:10:01.524391+0000","last_fullsized":"2024-04-24T18:10:01.524391+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:05:35.105903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"153'928","reported_seq":10039,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.551359+0000","last_change":"2024-04-24T18:10:01.468161+0000","last_active":"2024-04-24T18:10:01.551359+0000","last_peered":"2024-04-24T18:10:01.551359+0000","last_clean":"2024-04-24T18:10:01.551359+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:10:01.551359+0000","last_undegraded":"2024-04-24T18:10:01.551359+0000","last_fullsized":"2024-04-24T18:10:01.551359+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":928,"log_dups_size":0,"ondisk_log_size":928,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001496982,"stat_sum":{"num_bytes":14741343,"num_objects":92,"num_object_clones":5,"num_object_copies":184,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":92,"num_whiteouts":76,"num_read":11806,"num_read_kb":43779,"num_write":1059,"num_write_kb":32713,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"f","length":"a"}]},{"pgid":"3.0","version":"0'0","reported_seq":117,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.464809+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:10:01.464809+0000","last_peered":"2024-04-24T18:10:01.464809+0000","last_clean":"2024-04-24T18:10:01.464809+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:10:01.464809+0000","last_undegraded":"2024-04-24T18:10:01.464809+0000","last_fullsized":"2024-04-24T18:10:01.464809+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":145,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.524099+0000","last_change":"2024-04-24T18:07:20.407800+0000","last_active":"2024-04-24T18:10:01.524099+0000","last_peered":"2024-04-24T18:10:01.524099+0000","last_clean":"2024-04-24T18:10:01.524099+0000","last_became_active":"2024-04-24T18:07:20.407560+0000","last_became_peered":"2024-04-24T18:07:20.407560+0000","last_unstale":"2024-04-24T18:10:01.524099+0000","last_undegraded":"2024-04-24T18:10:01.524099+0000","last_fullsized":"2024-04-24T18:10:01.524099+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:58:33.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"153'532","reported_seq":5147,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.314386+0000","last_change":"2024-04-24T18:10:02.314708+0000","last_active":"2024-04-24T18:10:02.314386+0000","last_peered":"2024-04-24T18:10:02.314386+0000","last_clean":"2024-04-24T18:10:02.314386+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:02.314386+0000","last_undegraded":"2024-04-24T18:10:02.314386+0000","last_fullsized":"2024-04-24T18:10:02.314386+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":532,"log_dups_size":0,"ondisk_log_size":532,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001097703,"stat_sum":{"num_bytes":7639501,"num_objects":63,"num_object_clones":8,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":50,"num_read":6047,"num_read_kb":29292,"num_write":623,"num_write_kb":15067,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"3.1","version":"113'2","reported_seq":145,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.463420+0000","last_change":"2024-04-24T18:07:27.503483+0000","last_active":"2024-04-24T18:10:01.463420+0000","last_peered":"2024-04-24T18:10:01.463420+0000","last_clean":"2024-04-24T18:10:01.463420+0000","last_became_active":"2024-04-24T18:07:27.503170+0000","last_became_peered":"2024-04-24T18:07:27.503170+0000","last_unstale":"2024-04-24T18:10:01.463420+0000","last_undegraded":"2024-04-24T18:10:01.463420+0000","last_fullsized":"2024-04-24T18:10:01.463420+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:59:30.992102+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":343,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.524470+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:10:01.524470+0000","last_peered":"2024-04-24T18:10:01.524470+0000","last_clean":"2024-04-24T18:10:01.524470+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:10:01.524470+0000","last_undegraded":"2024-04-24T18:10:01.524470+0000","last_fullsized":"2024-04-24T18:10:01.524470+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":132,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.464772+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:10:01.464772+0000","last_peered":"2024-04-24T18:10:01.464772+0000","last_clean":"2024-04-24T18:10:01.464772+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:10:01.464772+0000","last_undegraded":"2024-04-24T18:10:01.464772+0000","last_fullsized":"2024-04-24T18:10:01.464772+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"153'1015","reported_seq":9891,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.316192+0000","last_change":"2024-04-24T18:10:02.316416+0000","last_active":"2024-04-24T18:10:02.316192+0000","last_peered":"2024-04-24T18:10:02.316192+0000","last_clean":"2024-04-24T18:10:02.316192+0000","last_became_active":"2024-04-24T18:07:59.631859+0000","last_became_peered":"2024-04-24T18:07:59.631859+0000","last_unstale":"2024-04-24T18:10:02.316192+0000","last_undegraded":"2024-04-24T18:10:02.316192+0000","last_fullsized":"2024-04-24T18:10:02.316192+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1015,"log_dups_size":0,"ondisk_log_size":1015,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:14.465536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0026983889999999998,"stat_sum":{"num_bytes":18492280,"num_objects":65,"num_object_clones":7,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":51,"num_read":12941,"num_read_kb":32697,"num_write":1174,"num_write_kb":22152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.1","version":"154'653","reported_seq":6061,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.331152+0000","last_change":"2024-04-24T18:10:02.331414+0000","last_active":"2024-04-24T18:10:02.331152+0000","last_peered":"2024-04-24T18:10:02.331152+0000","last_clean":"2024-04-24T18:10:02.331152+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:02.331152+0000","last_undegraded":"2024-04-24T18:10:02.331152+0000","last_fullsized":"2024-04-24T18:10:02.331152+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":653,"log_dups_size":0,"ondisk_log_size":653,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":4,"snaptrim_duration":0.0064577310000000004,"stat_sum":{"num_bytes":9585993,"num_objects":77,"num_object_clones":2,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":72,"num_read":6928,"num_read_kb":29932,"num_write":547,"num_write_kb":23518,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"1.4","version":"0'0","reported_seq":146,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.521240+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:10:01.521240+0000","last_peered":"2024-04-24T18:10:01.521240+0000","last_clean":"2024-04-24T18:10:01.521240+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:01.521240+0000","last_undegraded":"2024-04-24T18:10:01.521240+0000","last_fullsized":"2024-04-24T18:10:01.521240+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"152'800","reported_seq":6047,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.320787+0000","last_change":"2024-04-24T18:10:02.321219+0000","last_active":"2024-04-24T18:10:02.320787+0000","last_peered":"2024-04-24T18:10:02.320787+0000","last_clean":"2024-04-24T18:10:02.320787+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:02.320787+0000","last_undegraded":"2024-04-24T18:10:02.320787+0000","last_fullsized":"2024-04-24T18:10:02.320787+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":800,"log_dups_size":0,"ondisk_log_size":800,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0037601660000000001,"stat_sum":{"num_bytes":17109279,"num_objects":62,"num_object_clones":5,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":49,"num_read":7379,"num_read_kb":18350,"num_write":831,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"1.5","version":"0'0","reported_seq":146,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.469000+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:10:01.469000+0000","last_peered":"2024-04-24T18:10:01.469000+0000","last_clean":"2024-04-24T18:10:01.469000+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:01.469000+0000","last_undegraded":"2024-04-24T18:10:01.469000+0000","last_fullsized":"2024-04-24T18:10:01.469000+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"151'660","reported_seq":11951,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.314164+0000","last_change":"2024-04-24T18:10:02.314433+0000","last_active":"2024-04-24T18:10:02.314164+0000","last_peered":"2024-04-24T18:10:02.314164+0000","last_clean":"2024-04-24T18:10:02.314164+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:10:02.314164+0000","last_undegraded":"2024-04-24T18:10:02.314164+0000","last_fullsized":"2024-04-24T18:10:02.314164+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":660,"log_dups_size":0,"ondisk_log_size":660,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049965999999999997,"stat_sum":{"num_bytes":12452441,"num_objects":77,"num_object_clones":6,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":68,"num_read":12469,"num_read_kb":39612,"num_write":635,"num_write_kb":18767,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.9","version":"153'636","reported_seq":6188,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.324753+0000","last_change":"2024-04-24T18:10:02.325343+0000","last_active":"2024-04-24T18:10:02.324753+0000","last_peered":"2024-04-24T18:10:02.324753+0000","last_clean":"2024-04-24T18:10:02.324753+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:02.324753+0000","last_undegraded":"2024-04-24T18:10:02.324753+0000","last_fullsized":"2024-04-24T18:10:02.324753+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":636,"log_dups_size":0,"ondisk_log_size":636,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00083609600000000004,"stat_sum":{"num_bytes":13740076,"num_objects":76,"num_object_clones":6,"num_object_copies":152,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":76,"num_whiteouts":63,"num_read":6090,"num_read_kb":18046,"num_write":652,"num_write_kb":12791,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.a","version":"153'679","reported_seq":3832,"reported_epoch":153,"state":"active+clean","last_fresh":"2024-04-24T18:10:01.468321+0000","last_change":"2024-04-24T18:10:01.468321+0000","last_active":"2024-04-24T18:10:01.468321+0000","last_peered":"2024-04-24T18:10:01.468321+0000","last_clean":"2024-04-24T18:10:01.468321+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:01.468321+0000","last_undegraded":"2024-04-24T18:10:01.468321+0000","last_fullsized":"2024-04-24T18:10:01.468321+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":679,"log_dups_size":0,"ondisk_log_size":679,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014971069999999999,"stat_sum":{"num_bytes":10175762,"num_objects":65,"num_object_clones":3,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":57,"num_read":3788,"num_read_kb":11579,"num_write":752,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[{"start":"f","length":"a"}]},{"pgid":"4.b","version":"154'677","reported_seq":5787,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.319084+0000","last_change":"2024-04-24T18:10:02.319319+0000","last_active":"2024-04-24T18:10:02.319084+0000","last_peered":"2024-04-24T18:10:02.319084+0000","last_clean":"2024-04-24T18:10:02.319084+0000","last_became_active":"2024-04-24T18:07:56.621225+0000","last_became_peered":"2024-04-24T18:07:56.621225+0000","last_unstale":"2024-04-24T18:10:02.319084+0000","last_undegraded":"2024-04-24T18:10:02.319084+0000","last_fullsized":"2024-04-24T18:10:02.319084+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":677,"log_dups_size":0,"ondisk_log_size":677,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:23:34.749779+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0059932819999999999,"stat_sum":{"num_bytes":5407814,"num_objects":53,"num_object_clones":1,"num_object_copies":106,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":53,"num_whiteouts":49,"num_read":8572,"num_read_kb":17236,"num_write":683,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":21073,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.c","version":"151'566","reported_seq":6975,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.332661+0000","last_change":"2024-04-24T18:10:02.333100+0000","last_active":"2024-04-24T18:10:02.332661+0000","last_peered":"2024-04-24T18:10:02.332661+0000","last_clean":"2024-04-24T18:10:02.332661+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:10:02.332661+0000","last_undegraded":"2024-04-24T18:10:02.332661+0000","last_fullsized":"2024-04-24T18:10:02.332661+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":566,"log_dups_size":0,"ondisk_log_size":566,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0075635260000000001,"stat_sum":{"num_bytes":11483954,"num_objects":63,"num_object_clones":5,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":54,"num_read":7628,"num_read_kb":28741,"num_write":579,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.d","version":"154'621","reported_seq":9185,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.332334+0000","last_change":"2024-04-24T18:10:02.332591+0000","last_active":"2024-04-24T18:10:02.332334+0000","last_peered":"2024-04-24T18:10:02.332334+0000","last_clean":"2024-04-24T18:10:02.332334+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:02.332334+0000","last_undegraded":"2024-04-24T18:10:02.332334+0000","last_fullsized":"2024-04-24T18:10:02.332334+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0076172269999999999,"stat_sum":{"num_bytes":5865782,"num_objects":78,"num_object_clones":5,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":70,"num_read":8966,"num_read_kb":23944,"num_write":545,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.e","version":"154'559","reported_seq":6979,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.336255+0000","last_change":"2024-04-24T18:10:02.336498+0000","last_active":"2024-04-24T18:10:02.336255+0000","last_peered":"2024-04-24T18:10:02.336255+0000","last_clean":"2024-04-24T18:10:02.336255+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:02.336255+0000","last_undegraded":"2024-04-24T18:10:02.336255+0000","last_fullsized":"2024-04-24T18:10:02.336255+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":559,"log_dups_size":0,"ondisk_log_size":559,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.010683405,"stat_sum":{"num_bytes":4048896,"num_objects":59,"num_object_clones":6,"num_object_copies":118,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":59,"num_whiteouts":49,"num_read":6635,"num_read_kb":16242,"num_write":552,"num_write_kb":13540,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.f","version":"154'855","reported_seq":6330,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.360981+0000","last_change":"2024-04-24T18:10:02.317306+0000","last_active":"2024-04-24T18:10:02.360981+0000","last_peered":"2024-04-24T18:10:02.360981+0000","last_clean":"2024-04-24T18:10:02.360981+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:10:02.360981+0000","last_undegraded":"2024-04-24T18:10:02.360981+0000","last_fullsized":"2024-04-24T18:10:02.360981+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":855,"log_dups_size":0,"ondisk_log_size":855,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.00397373,"stat_sum":{"num_bytes":5238836,"num_objects":67,"num_object_clones":2,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":60,"num_read":8326,"num_read_kb":20866,"num_write":910,"num_write_kb":8626,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.10","version":"154'644","reported_seq":7386,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.321107+0000","last_change":"2024-04-24T18:10:02.321107+0000","last_active":"2024-04-24T18:10:02.321107+0000","last_peered":"2024-04-24T18:10:02.321107+0000","last_clean":"2024-04-24T18:10:02.321107+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:02.321107+0000","last_undegraded":"2024-04-24T18:10:02.321107+0000","last_fullsized":"2024-04-24T18:10:02.321107+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":644,"log_dups_size":0,"ondisk_log_size":644,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":3,"snaptrim_duration":0.0039189209999999997,"stat_sum":{"num_bytes":3378675,"num_objects":77,"num_object_clones":9,"num_object_copies":154,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":77,"num_whiteouts":67,"num_read":8160,"num_read_kb":18559,"num_write":660,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.11","version":"154'728","reported_seq":9008,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.324416+0000","last_change":"2024-04-24T18:10:02.324714+0000","last_active":"2024-04-24T18:10:02.324416+0000","last_peered":"2024-04-24T18:10:02.324416+0000","last_clean":"2024-04-24T18:10:02.324416+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:10:02.324416+0000","last_undegraded":"2024-04-24T18:10:02.324416+0000","last_fullsized":"2024-04-24T18:10:02.324416+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":728,"log_dups_size":0,"ondisk_log_size":728,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0072059890000000003,"stat_sum":{"num_bytes":10565948,"num_objects":78,"num_object_clones":5,"num_object_copies":156,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":78,"num_whiteouts":67,"num_read":9393,"num_read_kb":48646,"num_write":843,"num_write_kb":30861,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.12","version":"151'623","reported_seq":6333,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.315179+0000","last_change":"2024-04-24T18:10:02.315438+0000","last_active":"2024-04-24T18:10:02.315179+0000","last_peered":"2024-04-24T18:10:02.315179+0000","last_clean":"2024-04-24T18:10:02.315179+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:02.315179+0000","last_undegraded":"2024-04-24T18:10:02.315179+0000","last_fullsized":"2024-04-24T18:10:02.315179+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":623,"log_dups_size":0,"ondisk_log_size":623,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00147444,"stat_sum":{"num_bytes":10008013,"num_objects":86,"num_object_clones":5,"num_object_copies":172,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":86,"num_whiteouts":73,"num_read":10220,"num_read_kb":31576,"num_write":562,"num_write_kb":8705,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.13","version":"154'958","reported_seq":9057,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.334172+0000","last_change":"2024-04-24T18:10:02.334172+0000","last_active":"2024-04-24T18:10:02.334172+0000","last_peered":"2024-04-24T18:10:02.334172+0000","last_clean":"2024-04-24T18:10:02.334172+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:10:02.334172+0000","last_undegraded":"2024-04-24T18:10:02.334172+0000","last_fullsized":"2024-04-24T18:10:02.334172+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":958,"log_dups_size":0,"ondisk_log_size":958,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":2,"snaptrim_duration":0.0090748879999999997,"stat_sum":{"num_bytes":4584004,"num_objects":54,"num_object_clones":3,"num_object_copies":108,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":54,"num_whiteouts":48,"num_read":10393,"num_read_kb":22683,"num_write":1288,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.14","version":"154'655","reported_seq":7841,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.384359+0000","last_change":"2024-04-24T18:10:02.324233+0000","last_active":"2024-04-24T18:10:02.384359+0000","last_peered":"2024-04-24T18:10:02.384359+0000","last_clean":"2024-04-24T18:10:02.384359+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:10:02.384359+0000","last_undegraded":"2024-04-24T18:10:02.384359+0000","last_fullsized":"2024-04-24T18:10:02.384359+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":655,"log_dups_size":0,"ondisk_log_size":655,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0063674129999999997,"stat_sum":{"num_bytes":3869283,"num_objects":50,"num_object_clones":2,"num_object_copies":100,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":50,"num_whiteouts":43,"num_read":9676,"num_read_kb":25547,"num_write":546,"num_write_kb":9846,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.15","version":"150'603","reported_seq":5946,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.314272+0000","last_change":"2024-04-24T18:10:02.314569+0000","last_active":"2024-04-24T18:10:02.314272+0000","last_peered":"2024-04-24T18:10:02.314272+0000","last_clean":"2024-04-24T18:10:02.314272+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:10:02.314272+0000","last_undegraded":"2024-04-24T18:10:02.314272+0000","last_fullsized":"2024-04-24T18:10:02.314272+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":603,"log_dups_size":0,"ondisk_log_size":603,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00050144199999999995,"stat_sum":{"num_bytes":4418270,"num_objects":69,"num_object_clones":8,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":59,"num_read":7631,"num_read_kb":17555,"num_write":496,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[{"start":"e","length":"6"}]},{"pgid":"4.16","version":"152'1129","reported_seq":6451,"reported_epoch":154,"state":"active+clean","last_fresh":"2024-04-24T18:10:02.314820+0000","last_change":"2024-04-24T18:10:02.315660+0000","last_active":"2024-04-24T18:10:02.314820+0000","last_peered":"2024-04-24T18:10:02.314820+0000","last_clean":"2024-04-24T18:10:02.314820+0000","last_became_active":"2024-04-24T18:07:36.552566+0000","last_became_peered":"2024-04-24T18:07:36.552566+0000","last_unstale":"2024-04-24T18:10:02.314820+0000","last_undegraded":"2024-04-24T18:10:02.314820+0000","last_fullsized":"2024-04-24T18:10:02.314820+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1129,"log_dups_size":0,"ondisk_log_size":1129,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:06:29.992554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0022309809999999999,"stat_sum":{"num_bytes":7537269,"num_objects":75,"num_object_clones":4,"num_object_copies":150,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":67,"num_read":8173,"num_read_kb":32647,"num_write":1467,"num_write_kb":16911,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[{"start":"e","length":"6"}]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":295014878,"num_objects":2257,"num_object_clones":167,"num_object_copies":4514,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2257,"num_whiteouts":1933,"num_read":290371,"num_read_kb":859497,"num_write":28593,"num_write_kb":531601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1027,"num_bytes_recovered":43151386,"num_keys_recovered":150,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":18579456,"data_stored":136015534,"data_compressed":3595399,"data_compressed_allocated":9842688,"data_compressed_original":128854567,"omap_allocated":9738,"internal_metadata":0},"log_size":25384,"ondisk_log_size":25384,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":168,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":168,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":336,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5,"ondisk_log_size":5,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574884,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305092,"kb_used_data":3636,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94066748,"statfs":{"total":96636764160,"available":96324349952,"internally_reserved":0,"allocated":3723264,"data_stored":13315158,"data_compressed":306065,"data_compressed_allocated":962560,"data_compressed_original":11672173,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.48599999999999999,"15min":0.46800000000000003},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":2.7799999999999998,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.48899999999999999},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.48799999999999999,"15min":0.47399999999999998},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":2.585,"5min":2.585,"15min":2.585},"last":0.40999999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.84299999999999997,"5min":0.55100000000000005,"15min":0.503},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":7.6740000000000004,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.45900000000000002,"15min":0.45400000000000001},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":2.008,"5min":2.008,"15min":2.008},"last":0.437}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.80900000000000005,"5min":0.55300000000000005,"15min":0.51100000000000001},"min":{"1min":0.27300000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":7.3879999999999999,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.53500000000000003,"15min":0.499},"min":{"1min":0.308,"5min":0.307,"15min":0.307},"max":{"1min":7.4749999999999996,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.38700000000000001}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.53200000000000003,"15min":0.51400000000000001},"min":{"1min":0.29899999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.5499999999999998,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52300000000000002,"15min":0.52300000000000002},"min":{"1min":0.313,"5min":0.313,"15min":0.313},"max":{"1min":1.196,"5min":1.196,"15min":1.196},"last":0.495}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.24099999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.30199999999999999},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.498,"15min":0.502},"min":{"1min":0.25900000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":1.131,"5min":1.131,"15min":1.131},"last":0.35499999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.499,"15min":0.499},"min":{"1min":0.27400000000000002,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":2.2109999999999999,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.312,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6789999999999998,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.37}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.45400000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893135,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311080,"kb_used_data":4248,"kb_used_omap":31,"kb_used_meta":306784,"kb_avail":94060760,"statfs":{"total":96636764160,"available":96318218240,"internally_reserved":0,"allocated":4349952,"data_stored":24768331,"data_compressed":657902,"data_compressed_allocated":1703936,"data_compressed_original":23140608,"omap_allocated":32737,"internal_metadata":314146847},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.42499999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":10.071999999999999,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.499},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.442}]},{"osd":2,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.51600000000000001}]},{"osd":3,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.61699999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.57799999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.63400000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.67900000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574883,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306888,"kb_used_data":5436,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94064952,"statfs":{"total":96636764160,"available":96322510848,"internally_reserved":0,"allocated":5566464,"data_stored":21777285,"data_compressed":477374,"data_compressed_allocated":1449984,"data_compressed_original":19111936,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.59999999999999998,"15min":0.64000000000000001},"min":{"1min":0.19900000000000001,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":1.036,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.224},{"interface":"front","average":{"1min":0.377,"5min":1.0700000000000001,"15min":1.1859999999999999},"min":{"1min":0.191,"5min":0.191,"15min":0.191},"max":{"1min":0.70099999999999996,"5min":14.15,"15min":14.15},"last":0.35999999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.46200000000000002,"5min":0.499,"15min":0.505},"min":{"1min":0.23999999999999999,"5min":0.23999999999999999,"15min":0.23999999999999999},"max":{"1min":1.1799999999999999,"5min":1.306,"15min":1.306},"last":0.28199999999999997},{"interface":"front","average":{"1min":0.46800000000000003,"5min":1.135,"15min":1.246},"min":{"1min":0.217,"5min":0.217,"15min":0.217},"max":{"1min":1.2010000000000001,"5min":14.118,"15min":14.118},"last":0.311}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":1.1000000000000001,"15min":1.208},"min":{"1min":0.34699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.65900000000000003,"5min":14.089,"15min":14.089},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.40899999999999997,"5min":0.65700000000000003,"15min":0.69799999999999995},"min":{"1min":0.32500000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.60899999999999999,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.252}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.59899999999999998,"15min":0.61899999999999999},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.96699999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.622,"5min":1.23,"15min":1.331},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.8240000000000001,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.42899999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47899999999999998,"5min":0.73599999999999999,"15min":0.77900000000000003},"min":{"1min":0.31900000000000001,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.69199999999999995,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.371},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.65900000000000003,"15min":0.67700000000000005},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.032,"5min":1.49,"15min":1.49},"last":0.33600000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.441},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.495}]},{"osd":7,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":1.2270000000000001,"15min":1.345},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.84799999999999998,"5min":14.051,"15min":14.051},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.79700000000000004,"15min":0.84099999999999997},"min":{"1min":0.373,"5min":0.373,"15min":0.373},"max":{"1min":1.141,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.45300000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574884,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":304584,"kb_used_data":3120,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94067256,"statfs":{"total":96636764160,"available":96324870144,"internally_reserved":0,"allocated":3194880,"data_stored":11914875,"data_compressed":296656,"data_compressed_allocated":753664,"data_compressed_original":10474104,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.34499999999999997,"5min":0.40999999999999998,"15min":0.42099999999999999},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.67000000000000004,"5min":1.375,"15min":1.375},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.375,"5min":1.5389999999999999,"15min":1.7330000000000001},"min":{"1min":0.20200000000000001,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.54700000000000004,"5min":26.936,"15min":26.936},"last":0.315}]},{"osd":2,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":1.0960000000000001,"15min":1.206},"min":{"1min":0.26400000000000001,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":1.0629999999999999,"5min":11.93,"15min":11.93},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.40899999999999997,"15min":0.35599999999999998},"min":{"1min":0.26300000000000001,"5min":0.251,"15min":0.251},"max":{"1min":6.4889999999999999,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.36699999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.497,"5min":0.95899999999999996,"15min":1.036},"min":{"1min":0.32200000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.5880000000000001,"5min":14.371,"15min":14.371},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.98299999999999998,"15min":1.0129999999999999},"min":{"1min":0.30099999999999999,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":4.6390000000000002,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.40400000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":1.0580000000000001,"15min":1.1579999999999999},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.66700000000000004,"5min":14.414,"15min":14.414},"last":0.622},{"interface":"front","average":{"1min":0.51900000000000002,"5min":0.56799999999999995,"15min":0.57599999999999996},"min":{"1min":0.32000000000000001,"5min":0.317,"15min":0.317},"max":{"1min":0.74099999999999999,"5min":1.329,"15min":1.329},"last":0.48099999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":6.5629999999999997,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":1.0920000000000001,"15min":1.173},"min":{"1min":0.38500000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.52,"5min":14.452,"15min":14.452},"last":0.53600000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.58599999999999997}]},{"osd":7,"last update":"Wed Apr 24 18:09:21 2024","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.82799999999999996,"15min":0.82299999999999995},"min":{"1min":0.371,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":6.5300000000000002,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.624,"5min":1.1080000000000001,"15min":1.1879999999999999},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.5580000000000001,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574883,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":316184,"kb_used_data":2436,"kb_used_omap":1,"kb_used_meta":313726,"kb_avail":94055656,"statfs":{"total":96636764160,"available":96312991744,"internally_reserved":0,"allocated":2494464,"data_stored":4834034,"data_compressed":123472,"data_compressed_allocated":376832,"data_compressed_original":3674112,"omap_allocated":1704,"internal_metadata":321255768},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.443,"15min":0.41999999999999998},"min":{"1min":0.21199999999999999,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":4.109,"5min":4.109,"15min":4.109},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.44600000000000001,"15min":0.42199999999999999},"min":{"1min":0.23899999999999999,"5min":0.23899999999999999,"15min":0.23899999999999999},"max":{"1min":3.7410000000000001,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.308}]},{"osd":1,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.64700000000000002,"5min":0.44400000000000001,"15min":0.40999999999999998},"min":{"1min":0.253,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":4.8929999999999998,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.32700000000000001},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.40400000000000003,"15min":0.39200000000000002},"min":{"1min":0.26900000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.619,"5min":1.619,"15min":1.619},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.432,"15min":0.42999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.74299999999999999,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.52900000000000003,"15min":0.48699999999999999},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":4.1399999999999997,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.60999999999999999,"15min":0.57799999999999996},"min":{"1min":0.33200000000000002,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":5.3449999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.56000000000000005,"15min":0.56999999999999995},"min":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.88200000000000001,"5min":1.155,"15min":1.155},"last":0.45600000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.60499999999999998,"15min":0.59699999999999998},"min":{"1min":0.42399999999999999,"5min":0.372,"15min":0.372},"max":{"1min":2.9350000000000001,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.42599999999999999},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.63700000000000001,"15min":0.62},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":4.0679999999999996,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.49099999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.443},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.47899999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:09:18 2024","interfaces":[{"interface":"back","average":{"1min":1.145,"5min":0.72899999999999998,"15min":0.65900000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":10.516,"5min":10.516,"15min":10.516},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.63200000000000001,"15min":0.621},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":2.9620000000000002,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.52400000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574884,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":306728,"kb_used_data":5268,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94065112,"statfs":{"total":96636764160,"available":96322674688,"internally_reserved":0,"allocated":5394432,"data_stored":29299174,"data_compressed":719656,"data_compressed_allocated":2019328,"data_compressed_original":27080607,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.37,"15min":0.35799999999999998},"min":{"1min":0.20799999999999999,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.8119999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.33700000000000002},{"interface":"front","average":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.33000000000000002},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.27600000000000002},"max":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.25600000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.36099999999999999,"5min":0.41699999999999998,"15min":0.42599999999999999},"min":{"1min":0.221,"5min":0.221,"15min":0.221},"max":{"1min":1.2829999999999999,"5min":1.792,"15min":1.792},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.42099999999999999,"15min":0.40899999999999997},"min":{"1min":0.27100000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":2.8999999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.45800000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.38100000000000001,"15min":0.377},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.372},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.47299999999999998,"15min":0.44900000000000001},"min":{"1min":0.30099999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":4.6699999999999999,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.433}]},{"osd":4,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64300000000000002,"5min":0.57499999999999996,"15min":0.56299999999999994},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":3.2360000000000002,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":1.3129999999999999,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.40400000000000003}]},{"osd":5,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.59199999999999997,"15min":0.58399999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":3.2130000000000001,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.60099999999999998,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":3.4049999999999998,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.39400000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.442}]},{"osd":7,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.83599999999999997,"5min":0.63200000000000001,"15min":0.59799999999999998},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":3.8180000000000001,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":1.3100000000000001,"5min":0.75,"15min":0.65600000000000003},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":15.4,"5min":15.4,"15min":15.4},"last":0.503}]}]},{"osd":4,"up_from":13,"seq":55834574884,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":310576,"kb_used_data":5020,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94061264,"statfs":{"total":96636764160,"available":96318734336,"internally_reserved":0,"allocated":5140480,"data_stored":32345564,"data_compressed":886568,"data_compressed_allocated":2211840,"data_compressed_original":30459544,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47499999999999998,"15min":0.47699999999999998},"min":{"1min":0.28699999999999998,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.495,"15min":0.497},"min":{"1min":0.307,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.435}]},{"osd":1,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.49199999999999999,"15min":0.5},"min":{"1min":0.30399999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.65300000000000002,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.441,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.61599999999999999,"5min":0.627,"15min":0.627},"last":0.497}]},{"osd":2,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.93899999999999995,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.49299999999999999,"15min":0.497},"min":{"1min":0.26900000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.95399999999999996,"5min":1.153,"15min":1.153},"last":0.47099999999999997}]},{"osd":3,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.48399999999999999,"15min":0.48999999999999999},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.80600000000000005,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.52200000000000002,"15min":0.504},"min":{"1min":0.32700000000000001,"5min":0.32700000000000001,"15min":0.32700000000000001},"max":{"1min":2.5019999999999998,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.55600000000000005}]},{"osd":5,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.42399999999999999,"15min":0.42599999999999999},"min":{"1min":0.24199999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.39400000000000002,"5min":0.442,"15min":0.45000000000000001},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.64800000000000002,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.53700000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.51200000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:09:22 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.498,"15min":0.51000000000000001},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.83299999999999996,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.53600000000000003,"15min":0.54600000000000004},"min":{"1min":0.25600000000000001,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.877,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.59999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574884,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":305852,"kb_used_data":4388,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94065988,"statfs":{"total":96636764160,"available":96323571712,"internally_reserved":0,"allocated":4493312,"data_stored":14878334,"data_compressed":363170,"data_compressed_allocated":1011712,"data_compressed_original":12563914,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.42499999999999999,"15min":0.43099999999999999},"min":{"1min":0.26500000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.61199999999999999,"5min":0.67000000000000004,"15min":0.67000000000000004},"last":0.308},{"interface":"front","average":{"1min":0.434,"5min":0.46400000000000002,"15min":0.46899999999999997},"min":{"1min":0.251,"5min":0.222,"15min":0.222},"max":{"1min":0.64600000000000002,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.90400000000000003}]},{"osd":1,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.436,"15min":0.44},"min":{"1min":0.30499999999999999,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66600000000000004,"5min":0.72899999999999998,"15min":0.72899999999999998},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.44800000000000001,"5min":0.48799999999999999,"15min":0.495},"min":{"1min":0.28100000000000003,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.59499999999999997,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.81299999999999994}]},{"osd":2,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.52500000000000002,"15min":0.54400000000000004},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.52700000000000002,"5min":1.296,"15min":1.296},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44600000000000001,"5min":0.52500000000000002,"15min":0.53800000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.71999999999999997,"5min":1.377,"15min":1.377},"last":0.34100000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.621,"15min":0.64600000000000002},"min":{"1min":0.33700000000000002,"5min":0.313,"15min":0.313},"max":{"1min":0.68700000000000006,"5min":1.899,"15min":1.899},"last":0.623},{"interface":"front","average":{"1min":0.499,"5min":0.628,"15min":0.65000000000000002},"min":{"1min":0.35999999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.78200000000000003,"5min":1.881,"15min":1.881},"last":0.59899999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.375,"5min":0.49099999999999999,"15min":0.51000000000000001},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.52800000000000002,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.40799999999999997,"5min":0.54500000000000004,"15min":0.56799999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.74299999999999999,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.5}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.51800000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:09:20 2024","interfaces":[{"interface":"back","average":{"1min":0.41799999999999998,"5min":0.56000000000000005,"15min":0.58399999999999996},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.54900000000000004,"5min":1.016,"15min":1.016},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.42499999999999999,"5min":0.56499999999999995,"15min":0.58899999999999997},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.81200000000000006,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.67800000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":85,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":1347584,"data_stored":10979969,"data_compressed":296656,"data_compressed_allocated":753664,"data_compressed_original":10474104,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":3674112,"data_stored":20247010,"data_compressed":467672,"data_compressed_allocated":1413120,"data_compressed_original":18522112,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":3818895,"data_compressed":123472,"data_compressed_allocated":376832,"data_compressed_original":3674112,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":3264512,"data_stored":25926892,"data_compressed":683460,"data_compressed_allocated":1871872,"data_compressed_original":24766378,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":2945024,"data_stored":28444990,"data_compressed":750449,"data_compressed_allocated":1937408,"data_compressed_original":27551438,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":2629632,"data_stored":13887537,"data_compressed":362793,"data_compressed_allocated":1007616,"data_compressed_original":12510666,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":2396160,"data_stored":21855905,"data_compressed":617605,"data_compressed_allocated":1601536,"data_compressed_original":21174528,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":1757184,"data_stored":10854336,"data_compressed":293292,"data_compressed_allocated":880640,"data_compressed_original":10181229,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:10:04.239 INFO:tasks.thrashosds.thrasher:Setting 4.1f to [5, 2] 2024-04-24T18:10:04.239 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '4.1f', '5', '2'] 2024-04-24T18:10:04.240 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 4.1f 5 2 2024-04-24T18:10:04.313 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:10:04.371 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:04.380+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:04.410 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:04.411 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:04.411 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:04.411 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:04.421 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:10:04.464 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:04.473+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:04.533 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:04.542+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:04.546 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:04.546 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:04.546 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:04.546 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:04.546 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:04.556 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:10:04.641 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:04.650+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:04.723 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.211371467000006, 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.724 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.725 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:10:04.726 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.727 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:04.728 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.211350531999997, 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.729 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:10:04.730 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.731 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.732 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.211320774000001, 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.733 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:10:04.734 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:10:04.735 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:10:04.736 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207816280000003, 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.737 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.738 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.739 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.740 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207791946, 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.741 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:04.742 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:10:04.743 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:10:04.744 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207770355999997, 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:10:04.745 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.746 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.747 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.748 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.749 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207754230000006, 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.750 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:10:04.751 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.752 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.753 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207739337000007, 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.754 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.755 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.756 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.757 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207484570999995, 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:10:04.758 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.759 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.760 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:10:04.761 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207456776000001, 2024-04-24T18:10:04.762 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.763 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:10:04.764 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:10:04.765 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.766 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207424510999999, 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.767 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:04.768 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:10:04.769 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.770 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.207391587000004, 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.771 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.772 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.773 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:10:04.774 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.775 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205476488000002, 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:04.776 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.777 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.778 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.779 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205466479999998, 2024-04-24T18:10:04.780 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.781 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.782 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.783 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205453032999998, 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:10:04.784 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:04.785 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.786 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:10:04.787 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205444400000005, 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:10:04.788 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.789 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.790 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.791 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.792 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205435902999994, 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.793 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:10:04.794 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.795 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.796 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205428267000002, 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.797 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.798 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.799 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.800 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205420613000001, 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.801 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.802 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.803 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.804 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 65.205404063000003, 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.805 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:04.806 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:10:04.807 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:04.808 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:10:04.809 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:10:04.809 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.809 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.809 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.809 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:04.809 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:04.809 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:04.810 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:04.819+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:04.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:10:04.912 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:04.920+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:04.940 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:04.940 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:04.940 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:04.950 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:10:05.035 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:05.043+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:05.114 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:05.123+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:05.118 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:05.119 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:05.119 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:05.130 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:10:05.215 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:05.224+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:05.300 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 132.60214295599999, 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.305 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 128.68966814999999, 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.306 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:10:05.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262199627000001, 2024-04-24T18:10:05.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:10:05.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:10:05.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262178341999999, 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:10:05.316 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:10:05.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262160979000001, 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.320 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:10:05.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.323 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262115585999993, 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:10:05.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.327 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262088085000002, 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:10:05.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.331 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262069525000001, 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:05.332 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:10:05.333 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.334 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262045637, 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:10:05.335 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.336 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:10:05.337 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.338 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.262027743000004, 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.339 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.340 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.341 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.342 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261993101000002, 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:10:05.343 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.344 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:10:05.345 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.346 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261898572999996, 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.347 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.348 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.349 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.350 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261879891999996, 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.351 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:10:05.352 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:10:05.353 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.354 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261824412999999, 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.355 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:10:05.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261790027000004, 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.358 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261768067999995, 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.363 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261750985000006, 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:10:05.365 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:05.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:10:05.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:10:05.368 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261734524999994, 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.369 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:05.372 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261660610000007, 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 89.261608086999999, 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:10:05.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:05.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:05.379 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:05.380 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:05.381 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:10:05.382 INFO:teuthology.orchestra.run.smithi049.stderr:set 4.1f pg_upmap mapping to [5,2] 2024-04-24T18:10:05.383 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:05.389+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:05.482 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:05.491+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:05.490 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:10:05.555 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:05.555 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:05.555 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:05.566 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:10:05.583 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:05.591+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:05.640 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:05.658 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:05.677 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:05.684 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:05.693+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:05.698 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:05.698 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:05.698 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:05.707 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:10:05.715 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:05.734 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:05.754 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:05.773 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:05.774 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:05.775 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:05.775 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:05.785 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:05.794+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:05.844 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:05.845 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:05.855 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:10:05.886 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:05.894+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:05.983 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:05.984 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:05.986 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:05.994+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:05.994 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:10:06.086 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:06.095+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:06.122 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:06.123 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:06.133 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:10:06.188 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:06.197+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:06.261 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:06.261 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:06.271 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:10:06.289 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:06.297+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:06.330 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:10:06.390 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:06.398+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:06.445 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:06.446 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:06.457 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:10:06.502 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:06.511+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:06.592 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:06.600+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:06.592 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:06.592 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:06.603 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:10:06.694 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:06.703+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:06.773 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:06.774 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:06.784 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:10:06.828 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:06.837+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:06.895 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:06.904+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:06.923 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:06.923 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:06.933 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:10:06.996 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:07.005+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:07.097 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:07.105+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:07.097 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:07.097 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:07.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:10:07.197 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:07.206+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:07.272 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:07.272 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:07.282 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:10:07.312 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:10:07.323 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:07.331+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:07.399 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:07.407+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:07.467 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:07.467 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:07.477 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:10:07.521 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:07.530+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:07.601 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:07.609+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:07.611 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:07.611 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:07.621 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:10:07.705 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:07.714+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:07.785 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:07.785 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:07.786 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:10:07.795 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:10:07.830 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:07.838+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:07.903 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:07.911+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:07.931 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:07.950 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:07.960 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:07.961 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:07.967 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:07.970 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:10:07.986 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:08.004 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:08.014 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:08.023+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:08.023 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:08.043 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:08.062 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:08.063 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:08.063 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:08.063 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:08.063 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:08.063 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:08.064 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:08.064 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:08.064 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:08.097 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:08.097 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:08.097 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:08.097 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:08.097 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:08.105 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:08.113+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:08.107 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:10:08.207 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:08.214+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:110 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.662498263, 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080309730000000003, 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.272 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 110 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.273 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616295+0000", 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616294+0000", 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.274 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616300+0000", 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8390000000000003e-06 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616309+0000", 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.568e-06 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616313+0000", 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5719999999999996e-06 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.275 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619843+0000", 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035296899999999998 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619890+0000", 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7530000000000001e-05 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.619966+0000", 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6094999999999996e-05 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.276 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621214+0000", 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0012478680000000001 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624251+0000", 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0030367929999999999 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624292+0000", 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1371000000000001e-05 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.277 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624326+0000", 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3200000000000001e-05 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:111 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:08.278 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.66247384799999, 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080874320000000003, 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 111 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.279 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616319+0000", 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.280 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616323+0000", 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0400000000000003e-06 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616331+0000", 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.3890000000000005e-06 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616336+0000", 2024-04-24T18:10:08.281 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.921e-06 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620195+0000", 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003859052 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620274+0000", 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9380000000000002e-05 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.282 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620405+0000", 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000130137 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622159+0000", 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0017542059999999999 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624335+0000", 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021760960000000002 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.283 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624373+0000", 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8321000000000001e-05 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624407+0000", 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3330000000000001e-05 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.284 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:112 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.662450699, 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0081433419999999996, 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 112 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.285 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616342+0000", 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.286 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616402+0000", 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9667999999999999e-05 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616411+0000", 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4870000000000008e-06 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616416+0000", 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5190000000000003e-06 2024-04-24T18:10:08.287 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620690+0000", 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042740629999999998 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620738+0000", 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.816e-05 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.288 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.620816+0000", 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8677999999999994e-05 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622287+0000", 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0014707629999999999 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624414+0000", 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0021272299999999999 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.289 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624453+0000", 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8271e-05 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624486+0000", 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2938000000000002e-05 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.290 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.66237013099999, 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.291 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.292 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:10:08.293 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.294 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:10:08.295 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.296 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.662345321, 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.297 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.298 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:10:08.299 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.300 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:10:08.301 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.302 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.66232520899999, 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:10:08.303 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.304 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:10:08.305 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.306 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.307 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.308 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.309 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.66225454600001, 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.310 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.311 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.312 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:10:08.313 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:10:08.314 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.315 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:08.316 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.662242773, 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:08.317 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.318 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.319 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.320 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.321 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:10:08.322 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.24192094399999, 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.323 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.324 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:10:08.325 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.326 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:10:08.327 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.328 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.329 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.24187376200001, 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:10:08.330 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:08.331 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.332 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.333 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:10:08.334 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.335 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.336 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.24184864099999, 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.337 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.338 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.339 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:10:08.340 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.341 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:10:08.342 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.241796507, 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.343 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.344 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.344 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:10:08.344 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.344 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.344 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.344 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.345 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:08.346 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.347 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:10:08.348 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.349 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:10:08.350 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 167.24173304300001, 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.351 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.352 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:10:08.353 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.354 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.355 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:10:08.356 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:10:08.357 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 135.46823432599999, 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.358 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:10:08.359 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.360 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.361 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.362 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 135.46712825700001, 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:08.363 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.364 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.365 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:10:08.366 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:125912 4.0 e78/26 4:07f98ca8:::rbd_data.1174a200c953.000000000000000a:head v 78'212, mlcod=77'210)", 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 96.311956538999993, 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083383320000000004, 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.367 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966836+0000", 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.966835+0000", 2024-04-24T18:10:08.368 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970839+0000", 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00400395 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970853+0000", 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3447e-05 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.369 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970859+0000", 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.8190000000000003e-06 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970899+0000", 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9919999999999997e-05 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.970916+0000", 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6473999999999998e-05 2024-04-24T18:10:08.370 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975140+0000", 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0042238680000000004 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:31.975175+0000", 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5148999999999998e-05 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.371 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 68.250002594999998, 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.372 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.373 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:10:08.374 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.375 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199220 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'517, mlcod=102'514)", 2024-04-24T18:10:08.376 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 68.248715512000004, 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080224549999999995, 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:08.377 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030077+0000", 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030085+0000", 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9009999999999994e-06 2024-04-24T18:10:08.378 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030097+0000", 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2106999999999999e-05 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030103+0000", 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9220000000000002e-06 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.379 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030156+0000", 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3511999999999999e-05 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.030177+0000", 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0699e-05 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038078+0000", 2024-04-24T18:10:08.380 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0079009019999999996 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038100+0000", 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2178e-05 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.381 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 67.909539441000007, 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.382 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.383 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.384 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:10:08.385 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:10:08.386 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 67.909449746999996, 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.387 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.388 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.389 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.390 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.391 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.392 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:08.394 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:10:08.394 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:08.402+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:08.494 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:08.503+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:08.580 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:08.581 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:08.581 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:08.581 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:08.591 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:10:08.633 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:08.641+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:08.696 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:08.704+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:08.758 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:08.758 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:08.759 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:08.765 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:10:08.808 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:08.817+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.379518064999999, 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.891 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.892 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.893 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.894 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:08.895 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.379497130000004, 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.896 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.897 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:10:08.898 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.899 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.379467371999993, 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:08.900 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:10:08.901 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.902 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.903 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375962877999996, 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.904 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.905 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:10:08.906 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.907 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375938543999993, 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.908 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.909 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.910 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375916954000004, 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:08.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:10:08.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.915 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375900827999999, 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:10:08.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:10:08.919 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375885934999999, 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:10:08.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:08.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:10:08.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:10:08.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375631169000002, 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.924 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:10:08.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375603373999994, 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375571109000006, 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.932 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.375538184999996, 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.936 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:10:08.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373623085999995, 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.940 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:10:08.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:10:08.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373613078000005, 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.944 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:08.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:10:08.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:10:08.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373599631000005, 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.948 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:10:08.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373590997999997, 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:08.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:10:08.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:10:08.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373582501000001, 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:10:08.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373574864999995, 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:10:08.960 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:08.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:10:08.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:10:08.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373567210999994, 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:10:08.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:10:08.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 69.373550660999996, 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:08.968 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:08.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:08.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:08.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:08.972 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:08.974 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:08.982+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:08.974 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:10:09.075 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:09.083+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:09.106 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:09.106 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:09.106 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:09.116 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:10:09.176 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:09.184+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:09.245 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:09.245 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:09.245 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:09.245 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:09.245 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:09.255 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:10:09.298 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:09.306+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:09.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:10:09.377 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:09.385+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:09.386 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:09.386 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:09.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:09.386 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:09.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.386 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 136.68831955100001, 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:10:09.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:09.392 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 132.775844745, 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:09.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:10:09.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348376221999999, 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.398 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:09.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:10:09.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.401 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348354936999996, 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:10:09.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:09.405 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348337573999999, 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:10:09.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348292181000005, 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.34826468, 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.416 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348246119999999, 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:10:09.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:10:09.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348222231999998, 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:10:09.420 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348204338000002, 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.424 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.427 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348169695999999, 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:10:09.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:09.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:10:09.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.431 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348075167999994, 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.432 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.433 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.434 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.435 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348056486999994, 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.436 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.437 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:10:09.438 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.439 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.348001007999997, 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.440 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.441 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.442 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.443 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.347966622000001, 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.444 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:10:09.445 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:10:09.446 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.347944663000007, 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:10:09.447 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:09.448 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.449 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.450 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.347927580000004, 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.451 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.452 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:10:09.453 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:10:09.454 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.347911120000006, 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:10:09.455 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:09.456 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.457 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.458 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.347837205000005, 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.459 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.460 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:10:09.461 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.462 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 93.347784681999997, 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:09.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.466 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:09.467 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:10:09.511 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:09.519+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:09.578 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:09.586+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:09.621 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:09.621 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:09.621 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:09.621 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:09.632 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:10:09.679 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:09.688+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:09.780 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:09.788+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:09.800 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:09.800 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:09.800 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:09.800 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:09.800 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:09.810 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:10:09.896 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:09.904+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:09.981 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:09.981 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:09.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:09.981 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:09.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.981 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.517222417, 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.517204213, 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:10:09.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:09.989 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.51718279799999, 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:09.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.517165647, 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.993 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:10:09.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:09.997 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.51714947000001, 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:10:09.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:10.001 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.51713163300001, 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:10:10.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.51710103000001, 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.012 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:10:10.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:10:10.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.517080871, 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:10:10.016 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:10.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.020 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.517064594, 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:10:10.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.517047003, 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:10:10.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.51703113400001, 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.029 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:10.033 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.517012588, 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.035 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:10.037 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.51699654399999, 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.516979252, 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:10:10.041 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:10:10.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:10:10.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.516957596, 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.045 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.516938766, 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:10:10.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:10:10.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.516922431, 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:10:10.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:10:10.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.51690660600001, 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.057 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.060 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.516888182, 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:10:10.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:10:10.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 103.516871807, 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:10.064 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:10.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:10.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:10.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:10:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:10:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:10.068 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:10.068 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:10.069 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:10:10.070 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:10.077+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:10.074 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:10:10.170 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:10.178+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:10.244 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:10.244 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:10.244 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:10.244 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:10.254 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:10:10.259 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:10.271 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:10.279+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:10.277 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:10.297 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:10.316 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:10.335 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:10.354 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:10.372 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:10.380+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:10.373 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:10.383 INFO:tasks.thrashosds.thrasher:in_osds: [0, 2, 3, 4, 5, 6, 7, 1] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:10:10.383 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:10:10.383 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 2, 3, 4, 5, 6, 7, 1] 2024-04-24T18:10:10.383 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-04-24T18:10:10.384 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:10.384 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:10.394 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:10.395 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:10.396 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:10.472 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:10.480+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:10.481 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:10:10.537 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:10.537 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:10.549 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:10:10.590 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:10.598+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:10.674 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:10.682+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:10.719 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:10.719 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:10.729 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:10:10.763 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-04-24T18:10:10.771+0000 7fa70e0ea700 -1 mon.a@0(leader).osd e162 definitely_dead 0 2024-04-24T18:10:10.774 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:10.782+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:10.858 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:10.858 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:10.868 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:10:10.875 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:10.884+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:10.977 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:10.985+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:11.036 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:11.036 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:11.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:10:11.078 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:11.086+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:11.178 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:11.187+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:11.184 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:11.184 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:11.194 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:10:11.279 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:11.288+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:11.359 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:11.359 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:11.369 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:10:11.380 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:11.388+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:11.481 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:11.490+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:11.487 INFO:teuthology.orchestra.run.smithi049.stderr:marked out osd.0. 2024-04-24T18:10:11.541 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:11.541 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:11.551 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:10:11.592 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:11.601+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:11.684 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:11.691+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:11.716 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:11.716 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:11.725 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:10:11.784 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:11.792+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:11.848 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:11.849 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:11.858 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:10:11.885 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:11.893+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:11.983 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:11.983 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:11.986 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:11.994+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:11.993 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:10:12.087 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:12.096+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:12.116 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:12.117 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:12.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:10:12.210 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:12.219+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:12.289 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:12.297+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:12.290 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:12.291 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:12.300 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:10:12.389 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:12.397+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:12.409 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:10:12.465 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:12.465 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:12.475 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:10:12.497 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-04-24T18:10:12.516 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:12.525+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:12.568 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:12.587 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:12.591 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:12.599+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:12.605 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:12.625 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:12.640 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:12.640 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:12.645 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:12.651 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:10:12.664 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:12.682 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:12.693 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:12.701+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:12.701 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:12.703 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:12.775 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:12.775 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:12.775 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:12.785 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:10:12.828 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:12.837+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:12.895 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:12.903+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.398586730000005, 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.910 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.911 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:10:12.912 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:10:12.913 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.398565794999996, 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.914 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.915 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.916 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.917 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.398536037, 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.918 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.919 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.920 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.921 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.395031543000002, 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.922 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.923 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:10:12.924 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.925 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:12.926 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.395007208999999, 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.928 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.930 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.394985618999996, 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:12.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:10:12.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.394969493000005, 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.938 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.394954600000005, 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.939 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:10:12.942 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.394699833999994, 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:10:12.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:10:12.944 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:10:12.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:10:12.946 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.394672039, 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:10:12.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:10:12.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:10:12.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:10:12.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.394639773999998, 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:10:12.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:12.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:10:12.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:10:12.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.394606850000002, 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:10:12.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:10:12.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:10:12.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392691751000001, 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.959 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392681742999997, 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.963 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:10:12.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:10:12.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392668295999997, 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.967 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392659663000003, 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.971 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392651165999993, 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.979 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392643530000001, 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.983 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392635876, 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.986 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:12.987 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 73.392619326000002, 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:10:12.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:12.991 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:12.993 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:10:12.995 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:13.003+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:13.096 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:13.104+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:13.122 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:13.122 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:13.122 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:13.122 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:13.132 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:10:13.217 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:13.225+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:13.298 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:13.306+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:13.302 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:13.302 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:13.302 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:13.302 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:13.302 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:13.316 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:10:13.318 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is set 2024-04-24T18:10:13.399 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:13.407+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:13.499 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:13.508+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:13.500 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 140.80178367900001, 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:10:13.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:10:13.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:10:13.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 136.889308873, 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:10:13.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:13.512 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461840350000003, 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:13.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.516 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461819065, 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:10:13.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:10:13.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461801702000002, 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.520 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461756308999995, 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461728808000004, 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.527 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.528 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:10:13.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461710248000003, 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.531 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:13.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:10:13.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.534 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461686360000002, 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:10:13.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:10:13.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461668466000006, 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:10:13.538 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.542 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461633824000003, 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.544 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.545 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461539295999998, 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.546 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.547 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.548 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.549 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461520614999998, 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.550 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:13.551 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:10:13.552 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.553 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461465136000001, 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.554 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.555 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.556 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.557 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461430750000005, 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.558 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:13.559 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.560 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:10:13.561 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461408790999997, 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.562 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.563 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.564 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:10:13.565 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461391707999994, 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:10:13.566 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.567 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.568 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.569 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461375247999996, 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.570 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.571 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:10:13.572 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.573 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461301332999994, 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.574 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:10:13.575 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:10:13.576 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 97.461248810000001, 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:13.577 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:13.578 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:10:13.579 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:13.580 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:10:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:10:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:13.581 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:13.581 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:13.582 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:10:13.600 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:13.609+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:13.701 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:13.709+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:13.723 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:13.723 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:13.723 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:13.723 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:13.733 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:10:13.802 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:13.811+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:13.863 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:13.864 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:13.864 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:13.874 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:10:13.903 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:13.912+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:14.004 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:14.012+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.544104249, 2024-04-24T18:10:14.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:10:14.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.544086045, 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.013 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54406462999999, 2024-04-24T18:10:14.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:10:14.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:10:14.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.021 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.544047479, 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:14.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:14.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.024 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54403130199999, 2024-04-24T18:10:14.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.026 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:10:14.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:10:14.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54401346500001, 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.030 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:10:14.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54398286200001, 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.035 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:10:14.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54396270300001, 2024-04-24T18:10:14.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:10:14.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54394642600001, 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:10:14.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:10:14.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:10:14.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543928835, 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.048 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.049 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:10:14.050 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:10:14.051 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.052 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54391296599999, 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:14.053 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.054 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.055 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.056 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54389442, 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.057 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.058 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:10:14.059 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:10:14.060 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543878376, 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.061 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.062 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.063 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.064 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543861084, 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.065 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.066 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.067 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.068 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.069 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543839428, 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.070 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.071 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:10:14.072 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:10:14.073 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543820598, 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.074 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:10:14.075 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.075 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.078 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.079 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.080 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:14.081 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543804263, 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.082 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.083 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:10:14.084 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.54378843800001, 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:10:14.085 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.086 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.087 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.088 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543770014, 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.089 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.090 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.091 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.092 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 107.543753639, 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.093 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.094 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.096 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:14.097 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:10:14.104 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:14.113+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:14.205 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:14.213+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:14.227 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:14.228 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:14.228 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:14.228 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:14.238 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:10:14.322 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:14.330+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:14.406 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:14.415+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:14.415 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:14.416 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:14.416 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:14.416 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:14.416 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:14.426 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:10:14.510 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:14.518+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:14.597 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:14.597 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:14.597 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:14.597 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467755948, 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.598 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.599 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:10:14.600 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:10:14.601 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46774140799999, 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.602 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.603 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.605 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46772655700001, 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:10:14.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:14.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:10:14.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467711408, 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46769662200001, 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.614 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:14.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:10:14.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:14.618 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467682033, 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:10:14.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:10:14.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.622 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467659208, 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:10:14.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46764355400001, 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.627 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467628432, 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.631 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:10:14.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.634 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467612616, 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:10:14.635 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:14.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:10:14.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.639 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467597614, 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:10:14.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.643 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46758275000001, 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.644 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:14.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467561633, 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.648 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:10:14.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:10:14.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467546403, 2024-04-24T18:10:14.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:14.656 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.467531666, 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.657 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:10:14.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.660 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46243045600001, 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:10:14.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:14.664 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46239461, 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.665 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.462374603, 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:10:14.668 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:10:14.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.460553412, 2024-04-24T18:10:14.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:14.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 122.46052813199999, 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:10:14.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:10:14.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:14.680 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:14.681 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:14.682 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:10:14.682 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:14.689+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:14.715 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:10:14.782 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:14.791+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:14.865 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:14.865 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:14.865 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:14.865 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:14.875 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:10:14.878 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:14.883 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:14.891+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:14.898 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:14.917 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:14.936 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:14.955 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:14.974 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:14.984 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:14.992+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:14.993 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:15.005 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:15.005 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:15.013 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:15.014 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:15.014 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:15.014 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:15.015 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:15.015 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:15.015 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:15.015 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:15.015 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:15.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:10:15.110 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:15.118+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:15.185 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:15.193+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:15.194 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:15.194 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:15.205 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:10:15.286 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:15.294+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:15.337 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-04-24T18:10:15.344 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:15.344 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:15.355 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:10:15.387 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:15.395+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:15.488 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:15.496+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:15.520 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:15.521 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:15.530 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:10:15.590 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:15.597+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:15.690 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:15.698+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:15.695 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:15.695 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:15.705 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:10:15.790 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:15.799+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:15.869 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:15.870 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:15.879 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:10:15.921 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:15.929+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:15.991 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:16.000+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:16.043 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:16.043 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:16.053 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:10:16.097 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:16.106+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:16.177 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:16.177 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:16.187 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:10:16.227 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:16.236+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:16.294 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:16.302+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:16.330 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is set 2024-04-24T18:10:16.354 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:16.354 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:16.365 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:10:16.407 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:16.415+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:16.496 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:16.504+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:16.499 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 7, 1] out_osds: [0] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:10:16.499 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:10:16.499 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-04-24T18:10:16.499 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-04-24T18:10:16.529 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:16.529 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:16.539 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:10:16.596 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:16.604+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:16.663 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:16.663 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:16.673 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:10:16.716 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:16.725+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:16.796 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:16.796 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:16.797 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:16.805+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:16.805 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:10:16.846 INFO:tasks.ceph.mon.a.smithi049.stderr:2024-04-24T18:10:16.853+0000 7fa70e0ea700 -1 mon.a@0(leader).osd e168 definitely_dead 0 2024-04-24T18:10:16.898 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:16.907+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:16.970 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:16.970 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:16.980 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:10:17.021 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:17.030+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:17.026 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:10:17.100 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:17.108+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:17.143 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:17.143 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:17.154 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:10:17.175 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:17.195 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:17.200 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:17.208+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:17.285 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:17.285 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:17.285 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:17.285 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:17.295 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:10:17.319 INFO:teuthology.orchestra.run.smithi049.stderr:marked in osd.0. 2024-04-24T18:10:17.319 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:17.327+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:17.331 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-04-24T18:10:17.403 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:17.411+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:17.414 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:17.433 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:17.449 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:17.449 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:17.449 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:17.449 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:17.449 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:17.452 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:17.459 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:10:17.471 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:17.492 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:17.503 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:17.512+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:17.511 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:17.513 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:31862 4.3 4:c74df0c7:::rbd_data.117432c990a0.0000000000000016:head [stat,delete] snapc a=[a] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e42)", 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 144.90581597799999, 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.067222341000000005, 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:17.604 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 31862 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705325+0000", 2024-04-24T18:10:17.605 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705324+0000", 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705327+0000", 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5450000000000002e-06 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.606 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705333+0000", 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0990000000000004e-06 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.705337+0000", 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9670000000000003e-06 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756065+0000", 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.050727807 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756153+0000", 2024-04-24T18:10:17.607 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8521000000000002e-05 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for subops from 5", 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.756463+0000", 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00030970200000000002 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.762570+0000", 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0061073940000000004 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "op_commit", 2024-04-24T18:10:17.608 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772493+0000", 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0099232790000000001 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "commit_sent", 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772520+0000", 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6349000000000001e-05 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:52.772547+0000", 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7073999999999999e-05 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.609 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:40348 4.6 4:673736f7:::rbd_header.1174f9704cae:head [watch reconnect cookie 139789845466992 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e46)", 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 140.99334117199999, 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.012268932, 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 40348 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.610 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617799+0000", 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617797+0000", 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999981 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.611 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617805+0000", 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4800000000000004e-06 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617826+0000", 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0942e-05 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.617834+0000", 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0770000000000008e-06 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.612 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618018+0000", 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00018448299999999999 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:56.618025+0000", 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8739999999999997e-06 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627808+0000", 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0097827850000001 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for rw locks", 2024-04-24T18:10:17.613 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.627919+0000", 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000111459 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628832+0000", 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00091258599999999995 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.628950+0000", 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000117483 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.614 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:07:57.630068+0000", 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0011187599999999999 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130812 4.16 4:6a6158c5:::rbd_data.11743af029fe.0000000000000090:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565872649, 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065996790999999999, 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.615 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130812 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045268+0000", 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.616 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045267+0000", 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045270+0000", 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7520000000000001e-06 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045277+0000", 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1330000000000001e-06 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.617 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045282+0000", 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111127+0000", 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065844886000000005 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111265+0000", 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000138026 2024-04-24T18:10:17.618 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130826 4.6 4:67a27501:::rbd_data.11743af029fe.000000000000009e:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565851364, 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065954187999999997, 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130826 2024-04-24T18:10:17.619 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045289+0000", 2024-04-24T18:10:17.620 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045291+0000", 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045298+0000", 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4969999999999998e-06 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045302+0000", 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9349999999999996e-06 2024-04-24T18:10:17.621 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.047324+0000", 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0020225159999999998 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111243+0000", 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063919034 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.622 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130834 4.16 4:6c86ed3b:::rbd_data.11743af029fe.00000000000000a6:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565834001, 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066077372999999995, 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130834 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:17.623 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045307+0000", 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045306+0000", 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045309+0000", 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5569999999999998e-06 2024-04-24T18:10:17.624 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045315+0000", 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3729999999999998e-06 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045319+0000", 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.3680000000000004e-06 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111303+0000", 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065983396 2024-04-24T18:10:17.625 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111384+0000", 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1034999999999999e-05 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130851 4.1e 4:7d957413:::rbd_data.11743af029fe.00000000000000b7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.56578860800001, 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066130300000000003, 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.626 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130851 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.627 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045352+0000", 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045355+0000", 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0149999999999999e-06 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.628 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045361+0000", 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8759999999999999e-06 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045374+0000", 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.2448e-05 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111409+0000", 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066035241999999994 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.629 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111482+0000", 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3122999999999998e-05 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130852 4.1e 4:7af95986:::rbd_data.11743af029fe.00000000000000b8:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565761107, 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066192056999999999, 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.630 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130852 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.631 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045379+0000", 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045382+0000", 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9289999999999998e-06 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045388+0000", 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.632 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045393+0000", 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6399999999999996e-06 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111502+0000", 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066109377999999996 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111571+0000", 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9172999999999997e-05 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.633 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130858 4.1e 4:7e2ca42e:::rbd_data.11743af029fe.00000000000000be:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565742547, 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066266620999999998, 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130858 2024-04-24T18:10:17.634 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045398+0000", 2024-04-24T18:10:17.635 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045405+0000", 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8350000000000004e-06 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045412+0000", 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.764e-06 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045417+0000", 2024-04-24T18:10:17.636 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2930000000000002e-06 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111592+0000", 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066175650000000003 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111665+0000", 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2445000000000002e-05 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.637 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130863 4.1e 4:7e669bae:::rbd_data.11743af029fe.00000000000000c3:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565718659, 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066330352999999995, 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130863 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.638 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045422+0000", 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045421+0000", 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.639 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045424+0000", 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.8609999999999998e-06 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045431+0000", 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5740000000000001e-06 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045435+0000", 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.143e-06 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.640 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111684+0000", 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066249274999999996 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111752+0000", 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7980999999999998e-05 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130865 4.16 4:687c7744:::rbd_data.11743af029fe.00000000000000c5:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:17.641 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565700765, 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066410705, 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130865 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.642 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045440+0000", 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045439+0000", 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045442+0000", 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4719999999999998e-06 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.643 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045448+0000", 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3890000000000002e-06 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045452+0000", 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9899999999999999e-06 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111774+0000", 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066322059000000003 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.644 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111850+0000", 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6200999999999998e-05 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130874 4.16 4:6829a7c1:::rbd_data.11743af029fe.00000000000000ce:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565666123, 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066482240999999997, 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.645 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130874 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:17.646 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045474+0000", 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045476+0000", 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045483+0000", 2024-04-24T18:10:17.647 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2280000000000003e-06 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045494+0000", 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1325000000000001e-05 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111874+0000", 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066379576999999995 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111957+0000", 2024-04-24T18:10:17.648 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3141999999999998e-05 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130888 4.1e 4:7f329de1:::rbd_data.11743af029fe.00000000000000dc:head [sparse-read 0~13191] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.56557159499999, 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066764320000000002, 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.649 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130888 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045569+0000", 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.650 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045568+0000", 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045572+0000", 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.0230000000000001e-06 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045579+0000", 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9929999999999998e-06 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.651 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045583+0000", 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1060000000000002e-06 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.111979+0000", 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066395984000000005 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112046+0000", 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7486999999999999e-05 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.652 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112333+0000", 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00028718 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130889 4.6 4:66761f44:::rbd_data.11743af029fe.00000000000000dd:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.56555291399999, 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066560346000000006, 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.653 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130889 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045588+0000", 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.654 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045587+0000", 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045590+0000", 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045595+0000", 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8760000000000002e-06 2024-04-24T18:10:17.655 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045602+0000", 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5030000000000002e-06 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112082+0000", 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066479685999999996 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112148+0000", 2024-04-24T18:10:17.656 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6301000000000001e-05 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130903 4.16 4:6ff39099:::rbd_data.11743af029fe.00000000000000eb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565497435, 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066603764999999995, 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.657 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130903 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:17.658 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045643+0000", 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045657+0000", 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3997e-05 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.659 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045664+0000", 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0890000000000002e-06 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045672+0000", 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5210000000000002e-06 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112182+0000", 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066509178000000002 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.660 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112247+0000", 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-05 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130904 4.16 4:6f8874e1:::rbd_data.11743af029fe.00000000000000ec:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565463049, 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066641038999999999, 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.661 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130904 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.662 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045677+0000", 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045684+0000", 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7220000000000002e-06 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045690+0000", 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.4300000000000003e-06 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.663 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045694+0000", 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0779999999999997e-06 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112263+0000", 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066568671999999995 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112319+0000", 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5497000000000001e-05 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.664 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130915 4.1e 4:7f11ddb7:::rbd_data.11743af029fe.00000000000000f7:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.56544108999999, 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066730315999999998, 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130915 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.665 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045699+0000", 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.666 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045702+0000", 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.537e-06 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045708+0000", 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3999999999999997e-06 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045712+0000", 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5999999999999998e-06 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.667 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112373+0000", 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066661705000000002 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112430+0000", 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6480999999999997e-05 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130918 4.16 4:6d89f460:::rbd_data.11743af029fe.00000000000000fa:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.668 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565424007, 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066732490000000005, 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130918 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.669 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045717+0000", 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045716+0000", 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045719+0000", 2024-04-24T18:10:17.670 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.599e-06 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045725+0000", 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0950000000000001e-06 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045729+0000", 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7639999999999999e-06 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112365+0000", 2024-04-24T18:10:17.671 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066636350999999996 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112449+0000", 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4104999999999995e-05 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130919 4.16 4:6a7342e9:::rbd_data.11743af029fe.00000000000000fb:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.56540754700001, 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066790232000000005, 2024-04-24T18:10:17.672 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130919 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.673 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045733+0000", 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045735+0000", 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3269999999999999e-06 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.674 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045741+0000", 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8900000000000004e-06 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045744+0000", 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.631e-06 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112461+0000", 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066716141000000007 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112523+0000", 2024-04-24T18:10:17.675 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2602999999999997e-05 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130958 4.16 4:684592db:::rbd_data.11743af029fe.0000000000000122:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.56533363200001, 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066856757000000003, 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.676 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130958 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045807+0000", 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.677 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045806+0000", 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045809+0000", 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.988e-06 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045816+0000", 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6980000000000001e-06 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.678 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045825+0000", 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0389999999999993e-06 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112575+0000", 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066749473000000004 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112664+0000", 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9116000000000005e-05 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:130965 4.6 4:63435ea7:::rbd_data.11743af029fe.0000000000000129:head [sparse-read 0~524288] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 101.565281109, 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066757881000000005, 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 130965 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:17.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045859+0000", 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045862+0000", 2024-04-24T18:10:17.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.1609999999999999e-06 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045869+0000", 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8700000000000003e-06 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.045874+0000", 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7280000000000003e-06 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:17.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112546+0000", 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066671859 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:36.112617+0000", 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1789e-05 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:17.683 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:17.684 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:17.693+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:17.685 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:10:17.786 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:17.794+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:17.814 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:17.814 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:17.814 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:17.814 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:17.824 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:10:17.887 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:17.895+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:17.953 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:17.954 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:17.954 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:17.954 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:17.954 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:17.964 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:10:17.987 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:17.996+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:18.088 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:18.096+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:18.093 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94512 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 884736~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629916885, 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056303625000000003, 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.094 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94512 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471475+0000", 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.095 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471474+0000", 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471477+0000", 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6240000000000002e-06 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471485+0000", 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5129999999999999e-06 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.096 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471489+0000", 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527557+0000", 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056068825000000003 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527647+0000", 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9380000000000001e-05 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.097 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527778+0000", 2024-04-24T18:10:18.098 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013163399999999999 2024-04-24T18:10:18.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.098 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.098 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.098 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.098 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.098 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629898681, 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.099 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.100 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.101 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:10:18.102 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62987726599999, 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.103 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.104 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.105 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.106 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.107 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629860115, 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.108 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.109 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.110 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.111 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62984393799999, 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.112 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.113 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.114 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:10:18.115 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62982610100001, 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.116 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:10:18.117 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:10:18.118 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:10:18.119 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62979549799999, 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:10:18.120 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.121 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.122 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.123 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.124 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62977533900001, 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.125 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.126 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.127 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.128 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62975906200001, 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:10:18.129 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.130 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.130 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.130 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:18.130 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.130 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.130 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.131 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.132 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:10:18.133 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629741471, 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.134 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.135 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.136 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.137 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.138 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62972560199999, 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.139 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.140 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.141 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.142 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629707056, 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.143 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.144 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.145 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62969101199999, 2024-04-24T18:10:18.146 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.147 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:10:18.148 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.149 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.150 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62967372, 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:10:18.151 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:18.152 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:10:18.153 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.154 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629652064, 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.155 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.156 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:10:18.157 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:10:18.158 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629633234, 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.159 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:18.160 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:10:18.161 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.162 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629616899, 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:18.163 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:10:18.164 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:10:18.165 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.166 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62960107399999, 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.167 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:18.168 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:10:18.169 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.62958265, 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:18.170 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.171 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.172 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.173 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 111.629566275, 2024-04-24T18:10:18.174 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.175 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.176 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.177 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.178 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:18.179 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:10:18.189 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:18.197+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:18.290 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:18.299+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:18.346 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:10:18.355 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:18.355 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:18.355 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:18.355 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:18.366 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:10:18.407 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:18.415+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:18.493 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:18.501+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:18.537 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:18.537 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:18.537 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:18.548 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:10:18.594 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:18.602+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:18.677 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548208234, 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.678 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.679 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:10:18.680 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:10:18.681 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.682 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.54819369400001, 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.683 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.684 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.685 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.686 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548178843, 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.687 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.688 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:10:18.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:10:18.690 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548163694, 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.691 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.692 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.693 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.694 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548148908, 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.695 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.696 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.697 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.698 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548134319, 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.699 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.701 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.702 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548111494, 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.703 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.704 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.705 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.54809584, 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.707 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.708 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:10:18.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:10:18.710 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.54808071799999, 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.711 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.712 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:10:18.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:10:18.714 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.54806490199999, 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:10:18.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:18.716 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.717 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.718 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:18.719 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.5480499, 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.721 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.723 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548035036, 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.724 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:10:18.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:10:18.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.727 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.548013919, 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:10:18.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.731 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.547998689, 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:18.732 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:10:18.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:10:18.734 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.735 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.547983952, 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.736 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.737 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.738 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.739 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.542882742, 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.741 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.742 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.542846896, 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:18.744 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:10:18.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:10:18.746 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.747 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.542826889, 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:18.748 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:10:18.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:10:18.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.751 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.54100569800001, 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.752 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:10:18.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:10:18.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:18.755 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 126.540980418, 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:18.759 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:18.761 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:10:18.761 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:18.768+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:18.862 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:18.870+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:18.931 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:18.931 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:18.931 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:18.941 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:10:18.985 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:18.993+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:19.063 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:19.072+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:19.068 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:19.068 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:19.069 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:19.069 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:19.078 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:10:19.164 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:19.172+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:19.249 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.249 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.077951774, 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.250 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.251 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.252 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.253 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.058312648, 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.254 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:10:19.255 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.256 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.05828092599999, 2024-04-24T18:10:19.257 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.258 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.259 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.260 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.058257509, 2024-04-24T18:10:19.261 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.262 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.263 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.264 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.058227436, 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:10:19.265 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.266 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.267 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.268 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.269 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.057182289, 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.270 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.271 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.272 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.273 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.057165058, 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:10:19.274 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.275 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:10:19.276 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:10:19.277 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.05714806, 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.278 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.279 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.280 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.281 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.057129379, 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.282 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:19.283 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:10:19.284 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:10:19.285 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.05711191100001, 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:10:19.286 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:10:19.287 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.288 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.289 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.05709578699999, 2024-04-24T18:10:19.290 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.291 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.292 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:10:19.293 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.294 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.057064681, 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.295 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.296 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:10:19.297 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:10:19.298 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.05704548, 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.299 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.300 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:10:19.301 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.302 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.057024438, 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.303 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:19.304 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:10:19.305 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.306 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.05626949000001, 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.307 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.308 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.309 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.310 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.05624298799999, 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.311 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.312 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.313 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:10:19.314 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.056225458, 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.315 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.316 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:10:19.317 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.318 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.054762298, 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.319 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:10:19.320 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.321 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:19.322 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.04292365000001, 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.323 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.324 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:10:19.325 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 127.04290730699999, 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:10:19.326 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:19.327 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:19.328 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.329 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:19.330 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:19.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:10:19.332 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:10:19.334 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:19.339+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:19.432 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:19.441+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:19.477 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:19.478 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:19.478 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:19.478 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:19.488 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:10:19.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-04-24T18:10:19.568 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:19.576+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:19.634 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:19.642+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:19.670 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:19.670 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:19.670 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:19.682 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:10:19.692 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:19.712 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:19.735 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:19.735 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:19.743+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:19.756 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:19.776 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:19.796 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:19.815 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:19.816 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:19.821 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:19.826 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:10:19.846 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:19.848 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:19.868 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:19.876+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:19.937 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:19.945+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:19.991 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:19.991 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:20.001 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:10:20.044 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:20.053+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:20.124 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:20.124 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:20.134 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:10:20.138 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:20.146+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:20.239 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:20.247+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:20.298 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:20.298 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:20.308 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:10:20.351 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:20.360+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:20.438 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:20.438 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:20.440 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:20.448+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:20.448 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:10:20.541 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:20.550+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:20.611 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:20.611 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:20.621 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:10:20.663 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:20.671+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:20.743 INFO:tasks.ceph.osd.4.smithi078.stderr:2024-04-24T18:10:20.752+0000 7fac6dc40700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 43489) UID: 0 2024-04-24T18:10:20.786 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:20.786 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:20.796 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:10:20.844 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:20.852+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:20.945 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:20.953+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:20.961 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:20.961 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:20.971 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:10:21.055 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:21.064+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:21.134 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:21.135 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:21.145 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:10:21.185 INFO:tasks.ceph.osd.5.smithi078.stderr:2024-04-24T18:10:21.194+0000 7f5a1b36a700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 43492) UID: 0 2024-04-24T18:10:21.247 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:21.255+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:21.309 INFO:teuthology.orchestra.run.smithi078.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:21.309 INFO:teuthology.orchestra.run.smithi078.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:21.320 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-04-24T18:10:21.340 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:10:21.364 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:21.372+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:21.449 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:21.457+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:21.465 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:21.475 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-04-24T18:10:21.559 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:21.567+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:21.644 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:21.644 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:21.650 INFO:tasks.ceph.osd.0.smithi049.stderr:2024-04-24T18:10:21.658+0000 7fe2e695b700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 43450) UID: 0 2024-04-24T18:10:21.655 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-04-24T18:10:21.752 INFO:tasks.ceph.osd.2.smithi049.stderr:2024-04-24T18:10:21.760+0000 7f4abc431700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 43464) UID: 0 2024-04-24T18:10:21.823 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:21.825 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:21.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-04-24T18:10:21.861 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-04-24T18:10:21.876 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:21.883+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:21.953 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:21.962+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:22.007 INFO:teuthology.orchestra.run.smithi049.stdout:ERROR: (22) Invalid argument 2024-04-24T18:10:22.008 INFO:teuthology.orchestra.run.smithi049.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-04-24T18:10:22.018 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-04-24T18:10:22.023 INFO:teuthology.orchestra.run.smithi049.stdout:osd.0: {} 2024-04-24T18:10:22.042 INFO:teuthology.orchestra.run.smithi049.stdout:osd.1: {} 2024-04-24T18:10:22.054 INFO:tasks.ceph.osd.1.smithi049.stderr:2024-04-24T18:10:22.062+0000 7fdf17a65700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 43457) UID: 0 2024-04-24T18:10:22.061 INFO:teuthology.orchestra.run.smithi049.stdout:osd.2: {} 2024-04-24T18:10:22.082 INFO:teuthology.orchestra.run.smithi049.stdout:osd.3: {} 2024-04-24T18:10:22.101 INFO:teuthology.orchestra.run.smithi049.stdout:osd.4: {} 2024-04-24T18:10:22.120 INFO:teuthology.orchestra.run.smithi049.stdout:osd.5: {} 2024-04-24T18:10:22.140 INFO:teuthology.orchestra.run.smithi049.stdout:osd.6: {} 2024-04-24T18:10:22.151 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:22.151 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:22.151 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:22.151 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:22.151 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:22.155 INFO:tasks.ceph.osd.7.smithi078.stderr:2024-04-24T18:10:22.164+0000 7fdce7edb700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 43497) UID: 0 2024-04-24T18:10:22.159 INFO:teuthology.orchestra.run.smithi049.stdout:osd.7: {} 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.0: osd_enable_op_tracker = '' 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.1: osd_enable_op_tracker = '' 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.2: osd_enable_op_tracker = '' 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.3: osd_enable_op_tracker = '' 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.4: osd_enable_op_tracker = '' 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.5: osd_enable_op_tracker = '' 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.6: osd_enable_op_tracker = '' 2024-04-24T18:10:22.161 INFO:teuthology.orchestra.run.smithi049.stderr:osd.7: osd_enable_op_tracker = '' 2024-04-24T18:10:22.162 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-04-24T18:10:22.256 INFO:tasks.ceph.osd.3.smithi049.stderr:2024-04-24T18:10:22.264+0000 7fdad7683700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 43465) UID: 0 2024-04-24T18:10:22.327 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:10:22.332 INFO:tasks.thrashosds.thrasher:in_osds: [2, 3, 4, 5, 6, 7, 1, 0] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 7, 6] 2024-04-24T18:10:22.332 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-04-24T18:10:22.332 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-04-24T18:10:22.333 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-04-24T18:10:22.351 INFO:teuthology.orchestra.run.smithi049.stdout:All operations completed A-OK! 2024-04-24T18:10:22.354 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94513 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 892928~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887527701, 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522579000000003, 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94513 2024-04-24T18:10:22.355 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471493+0000", 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.356 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471498+0000", 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.784e-06 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471505+0000", 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6560000000000003e-06 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471510+0000", 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7650000000000001e-06 2024-04-24T18:10:22.357 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527807+0000", 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056296925999999997 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.527890+0000", 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.3362000000000006e-05 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.358 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528016+0000", 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012539499999999999 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94514 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 901120~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887506286, 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056760636000000003, 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.359 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94514 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.360 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471514+0000", 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471517+0000", 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6259999999999999e-06 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471523+0000", 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5359999999999998e-06 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.361 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471527+0000", 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9249999999999997e-06 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528045+0000", 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056518110000000003 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528139+0000", 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3739000000000005e-05 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.362 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528275+0000", 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000136026 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94515 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 909312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887489135, 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995057000000002, 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.363 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94515 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471532+0000", 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.364 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471531+0000", 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471534+0000", 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471539+0000", 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.84e-06 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.365 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471543+0000", 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8589999999999998e-06 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528303+0000", 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056759428000000001 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528392+0000", 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9220000000000003e-05 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.366 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528527+0000", 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134673 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94516 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 917504~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887472958, 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057217944, 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.367 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94516 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471548+0000", 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.368 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471547+0000", 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471550+0000", 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3180000000000001e-06 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471555+0000", 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7729999999999996e-06 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.369 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471561+0000", 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.587e-06 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528555+0000", 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056994164999999999 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528641+0000", 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5971999999999996e-05 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528766+0000", 2024-04-24T18:10:22.370 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000124449 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94517 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 925696~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887455121, 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057435420000000001, 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94517 2024-04-24T18:10:22.371 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471566+0000", 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471565+0000", 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.372 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471568+0000", 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5629999999999999e-06 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471574+0000", 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9359999999999996e-06 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471578+0000", 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1300000000000003e-06 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.373 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528794+0000", 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057215996999999998 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.528875+0000", 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1446999999999995e-05 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529001+0000", 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012583599999999999 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.374 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94518 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 933888~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887424518, 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057648410999999997, 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94518 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.375 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471596+0000", 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471598+0000", 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.835e-06 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.376 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471607+0000", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6000000000000007e-06 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471611+0000", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529029+0000", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057417157000000003 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529110+0000", 2024-04-24T18:10:22.377 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0976999999999994e-05 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529245+0000", 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000134967 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94519 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 942080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887404359, 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057914183000000001, 2024-04-24T18:10:22.378 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94519 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.379 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471616+0000", 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471618+0000", 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3999999999999999e-06 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471625+0000", 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2099999999999998e-06 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.380 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471628+0000", 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7280000000000001e-06 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529284+0000", 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057655623000000003 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529384+0000", 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000100251 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.381 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529530+0000", 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014631099999999999 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94520 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 950272~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887388082, 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058118578999999997, 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94520 2024-04-24T18:10:22.382 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471633+0000", 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471632+0000", 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.383 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471635+0000", 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2850000000000001e-06 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471641+0000", 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3350000000000003e-06 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471645+0000", 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7510000000000002e-06 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.384 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529558+0000", 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057913776 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529635+0000", 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6132999999999998e-05 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529751+0000", 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000116612 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.385 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94521 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 958464~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887370491, 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058339414999999999, 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94521 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.386 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471650+0000", 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471652+0000", 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5069999999999999e-06 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.387 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471658+0000", 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.7810000000000002e-06 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471662+0000", 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.788e-06 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529778+0000", 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058116608 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529854+0000", 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5388000000000004e-05 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.388 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.529990+0000", 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013573400000000001 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94522 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 966656~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887354622, 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058610667999999998, 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.389 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94522 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.390 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471666+0000", 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471668+0000", 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1900000000000002e-06 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471675+0000", 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.267e-06 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.391 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471680+0000", 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6809999999999997e-06 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530033+0000", 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058353469999999998 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530126+0000", 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3008000000000001e-05 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530277+0000", 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00015034900000000001 2024-04-24T18:10:22.392 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94523 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 974848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887336076, 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059037812000000002, 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94523 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.393 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471685+0000", 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471684+0000", 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.394 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471687+0000", 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.419e-06 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471693+0000", 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2840000000000003e-06 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471696+0000", 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5190000000000001e-06 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.395 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530531+0000", 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058834600000000001 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530601+0000", 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.0191000000000006e-05 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530722+0000", 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000121138 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.396 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94524 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 983040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.88732003200001, 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059297447000000003, 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94524 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.397 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471701+0000", 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471700+0000", 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471703+0000", 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3080000000000002e-06 2024-04-24T18:10:22.398 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471709+0000", 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.5560000000000004e-06 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471713+0000", 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0620000000000002e-06 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530764+0000", 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059050817999999998 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.399 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530858+0000", 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4427999999999998e-05 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.530998+0000", 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00013960399999999999 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.400 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94525 4.12 4:4c4ec6b7:::rbd_data.11748c7976ea.0000000000000018:20 [read 991232~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.88730274, 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059570369999999997, 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94525 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.401 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471718+0000", 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471723+0000", 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8640000000000001e-06 2024-04-24T18:10:22.402 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471730+0000", 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.083e-06 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471733+0000", 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9539999999999998e-06 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531043+0000", 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059309270999999997 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.403 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531136+0000", 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3238000000000004e-05 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531288+0000", 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000152285 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.404 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94526 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.88728108399999, 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059588595000000001, 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94526 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.405 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471740+0000", 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471739+0000", 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471745+0000", 2024-04-24T18:10:22.406 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4029999999999998e-06 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471750+0000", 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6699999999999999e-06 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471754+0000", 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531321+0000", 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059566697000000002 2024-04-24T18:10:22.407 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531328+0000", 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4900000000000003e-06 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94527 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.88726225400001, 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059603454, 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.408 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94527 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.409 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471758+0000", 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471760+0000", 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.429e-06 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.410 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471767+0000", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.302e-06 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471771+0000", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7910000000000002e-06 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531352+0000", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059581680999999997 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531362+0000", 2024-04-24T18:10:22.411 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6029999999999994e-06 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94528 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.88724591899999, 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639822000000002, 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.412 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94528 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471775+0000", 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471774+0000", 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.413 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471777+0000", 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1160000000000002e-06 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471782+0000", 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8490000000000002e-06 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471786+0000", 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.7529999999999999e-06 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.414 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531403+0000", 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059617322 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531415+0000", 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1095e-05 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94529 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887230094, 2024-04-24T18:10:22.415 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059663556, 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94529 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.416 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471791+0000", 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471790+0000", 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471793+0000", 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471800+0000", 2024-04-24T18:10:22.417 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.593e-06 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471804+0000", 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2679999999999996e-06 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531444+0000", 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059639594999999997 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531454+0000", 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0074e-05 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.418 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94530 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.88721167, 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059699145000000002, 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94530 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.419 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471809+0000", 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.420 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471811+0000", 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.514e-06 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471817+0000", 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.088e-06 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471821+0000", 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8720000000000004e-06 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.421 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531497+0000", 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059675534000000002 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531508+0000", 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1443999999999999e-05 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:94531 4.12 4.ed637232 (undecoded) ondisk+read+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 115.887195295, 2024-04-24T18:10:22.422 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059731197999999999, 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 94531 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:22.423 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471825+0000", 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471827+0000", 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.424 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471833+0000", 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.471844+0000", 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0390999999999999e-05 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531546+0000", 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05970197 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.425 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:26.531557+0000", 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0910999999999999e-05 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:232229 4.6 4:67cae2ed:::rbd_mirroring:head [list-watchers] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected+supports_pool_eio e170)", 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:10:18.323977+0000", 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 4.0350436800000002, 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.0404600770000001, 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "reached pg", 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.426 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 232229 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.323977+0000", 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.323977+0000", 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.323975+0000", 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999976 2024-04-24T18:10:22.427 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.323983+0000", 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9920000000000007e-06 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.323997+0000", 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.4099e-05 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.324006+0000", 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1929999999999999e-06 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.428 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.324048+0000", 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2177e-05 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "waiting for peered", 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:18.324053+0000", 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.1189999999999996e-06 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:19.364316+0000", 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.040263312 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.429 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:10:19.364437+0000", 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120579 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.430 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:22.432 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:22.432 DEBUG:teuthology.run_tasks:Unwinding manager rbd_fsx 2024-04-24T18:10:22.441 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-04-24T18:10:22.455 INFO:tasks.thrashosds:joining thrashosds 2024-04-24T18:10:22.455 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-04-24T18:10:22.456 INFO:tasks.ceph.osd.6.smithi078.stderr:2024-04-24T18:10:22.440+0000 7f6ed53c6700 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 58116) UID: 0 2024-04-24T18:10:22.602 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:22.602 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:22.602 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:22.603 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:22.613 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-04-24T18:10:22.673 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:22.673 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:22.674 INFO:teuthology.orchestra.run.smithi049.stdout: "epoch": 174, 2024-04-24T18:10:22.674 INFO:teuthology.orchestra.run.smithi049.stdout: "fsid": "5dae32d6-9101-4f7b-a6ef-ca5186fee71b", 2024-04-24T18:10:22.674 INFO:teuthology.orchestra.run.smithi049.stdout: "created": "2024-04-24T18:07:07.515859+0000", 2024-04-24T18:10:22.674 INFO:teuthology.orchestra.run.smithi049.stdout: "modified": "2024-04-24T18:10:22.323824+0000", 2024-04-24T18:10:22.674 INFO:teuthology.orchestra.run.smithi049.stdout: "last_up_change": "2024-04-24T18:08:52.865354+0000", 2024-04-24T18:10:22.674 INFO:teuthology.orchestra.run.smithi049.stdout: "last_in_change": "2024-04-24T18:10:17.313880+0000", 2024-04-24T18:10:22.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-04-24T18:10:22.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_num": 5799936, 2024-04-24T18:10:22.675 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_set": [ 2024-04-24T18:10:22.675 INFO:teuthology.orchestra.run.smithi049.stdout: "pglog_hardlimit", 2024-04-24T18:10:22.675 INFO:teuthology.orchestra.run.smithi049.stdout: "purged_snapdirs", 2024-04-24T18:10:22.675 INFO:teuthology.orchestra.run.smithi049.stdout: "recovery_deletes", 2024-04-24T18:10:22.676 INFO:teuthology.orchestra.run.smithi049.stdout: "sortbitwise" 2024-04-24T18:10:22.676 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.676 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_version": 5, 2024-04-24T18:10:22.676 INFO:teuthology.orchestra.run.smithi049.stdout: "full_ratio": 0.94999998807907104, 2024-04-24T18:10:22.676 INFO:teuthology.orchestra.run.smithi049.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-04-24T18:10:22.676 INFO:teuthology.orchestra.run.smithi049.stdout: "nearfull_ratio": 0.85000002384185791, 2024-04-24T18:10:22.676 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_snapshot": "", 2024-04-24T18:10:22.677 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_max": 4, 2024-04-24T18:10:22.677 INFO:teuthology.orchestra.run.smithi049.stdout: "max_osd": 8, 2024-04-24T18:10:22.677 INFO:teuthology.orchestra.run.smithi049.stdout: "require_min_compat_client": "luminous", 2024-04-24T18:10:22.677 INFO:teuthology.orchestra.run.smithi049.stdout: "min_compat_client": "luminous", 2024-04-24T18:10:22.677 INFO:teuthology.orchestra.run.smithi049.stdout: "require_osd_release": "quincy", 2024-04-24T18:10:22.677 INFO:teuthology.orchestra.run.smithi049.stdout: "pools": [ 2024-04-24T18:10:22.678 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.678 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 1, 2024-04-24T18:10:22.678 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "rbd", 2024-04-24T18:10:22.678 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:18.850062+0000", 2024-04-24T18:10:22.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-04-24T18:10:22.678 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-04-24T18:10:22.679 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:22.679 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:22.679 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:22.679 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:22.679 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:22.679 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:22.680 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:22.680 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:22.680 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:22.680 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:22.680 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 8, 2024-04-24T18:10:22.680 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 8, 2024-04-24T18:10:22.681 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 8, 2024-04-24T18:10:22.681 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 8, 2024-04-24T18:10:22.681 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 8, 2024-04-24T18:10:22.681 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:22.681 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:22.681 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:22.681 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:22.682 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:22.682 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:22.682 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:22.682 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.682 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "21", 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "21", 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 19, 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:22.683 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [ 2024-04-24T18:10:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: 3 2024-04-24T18:10:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:10:22.684 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": 3, 2024-04-24T18:10:22.685 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": 3, 2024-04-24T18:10:22.685 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:10:22.685 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:22.685 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:10:22.685 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:22.685 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:22.686 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:22.686 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:22.686 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:22.686 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:22.686 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:22.686 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:10:22.687 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.687 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:10:22.687 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:10:22.687 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:22.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:22.687 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:22.688 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:22.688 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:22.688 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:10:22.688 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:22.688 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:22.688 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:22.689 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:10:22.689 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:22.689 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:10:22.689 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.689 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.689 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 2, 2024-04-24T18:10:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": ".mgr", 2024-04-24T18:10:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:19.353026+0000", 2024-04-24T18:10:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 1, 2024-04-24T18:10:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool", 2024-04-24T18:10:22.690 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:22.691 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:22.691 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:22.691 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:22.691 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:22.691 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:22.691 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:22.692 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:22.692 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:22.692 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:22.692 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 1, 2024-04-24T18:10:22.692 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 1, 2024-04-24T18:10:22.692 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 1, 2024-04-24T18:10:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 1, 2024-04-24T18:10:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 1, 2024-04-24T18:10:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:22.693 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:22.694 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:22.694 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:22.694 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.694 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "17", 2024-04-24T18:10:22.694 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-04-24T18:10:22.694 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T18:10:22.695 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T18:10:22.695 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:22.695 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:22.695 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:10:22.695 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 0, 2024-04-24T18:10:22.695 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:10:22.696 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:22.697 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:10:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:10:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:10:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:22.698 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "options": { 2024-04-24T18:10:22.699 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_max": 32, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_min": 1 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "mgr": {} 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 3, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "cache", 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:25.800822+0000", 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 9, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool,incomplete_clones", 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:22.700 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 4, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 4, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 4, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 4, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 4, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:22.701 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "25", 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "21", 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "21", 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "21", 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 0, 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 19, 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:22.702 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": 1, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "writeback", 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 250, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:22.703 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "bloom", 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "false_positive_probability": 0.050000000000000003, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "target_size": 0, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "seed": 0 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 60, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 8, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [ 2024-04-24T18:10:22.704 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: 1000000 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:22.705 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 4, 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_name": "pool_client.0", 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "create_time": "2024-04-24T18:07:35.513947+0000", 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags": 8193, 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "flags_names": "hashpspool,selfmanaged_snaps", 2024-04-24T18:10:22.706 INFO:teuthology.orchestra.run.smithi049.stdout: "type": 1, 2024-04-24T18:10:22.707 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 2, 2024-04-24T18:10:22.707 INFO:teuthology.orchestra.run.smithi049.stdout: "min_size": 1, 2024-04-24T18:10:22.707 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_rule": 0, 2024-04-24T18:10:22.707 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_count": 0, 2024-04-24T18:10:22.707 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_target": 0, 2024-04-24T18:10:22.707 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_barrier": 0, 2024-04-24T18:10:22.707 INFO:teuthology.orchestra.run.smithi049.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "object_hash": 2, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_autoscale_mode": "off", 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num": 32, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num": 32, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_placement_num_target": 32, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_target": 32, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_num_pending": 32, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "last_pg_merge_meta": { 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "source_pgid": "0.0", 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "ready_epoch": 0, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_started": 0, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "last_epoch_clean": 0, 2024-04-24T18:10:22.708 INFO:teuthology.orchestra.run.smithi049.stdout: "source_version": "0'0", 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "target_version": "0'0" 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "last_change": "172", 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend": "0", 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_prenautilus": "0", 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "last_force_op_resend_preluminous": "0", 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "auid": 0, 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_mode": "selfmanaged", 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_seq": 107, 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "snap_epoch": 172, 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "pool_snaps": [], 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps": "[]", 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_bytes": 0, 2024-04-24T18:10:22.709 INFO:teuthology.orchestra.run.smithi049.stdout: "quota_max_objects": 0, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tiers": [], 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "tier_of": -1, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "read_tier": -1, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "write_tier": -1, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_mode": "none", 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_bytes": 0, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "target_max_objects": 0, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_target_full_ratio_micro": 800000, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_flush_age": 0, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "cache_min_evict_age": 0, 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profile": "", 2024-04-24T18:10:22.710 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_params": { 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "none" 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_period": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_count": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "use_gmt_hitset": true, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min_read_recency_for_promote": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "min_write_recency_for_promote": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_grade_decay_rate": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "hit_set_search_last_n": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "grade_table": [], 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "stripe_width": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "expected_num_objects": 0, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "fast_read": false, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "options": {}, 2024-04-24T18:10:22.711 INFO:teuthology.orchestra.run.smithi049.stdout: "application_metadata": { 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "rbd": {} 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "osds": [ 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "24da2d2a-953e-4440-9f6d-fb19cbec9ca1", 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:22.712 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 173, 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6801", 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.713 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6803", 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6805", 2024-04-24T18:10:22.714 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6807", 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6813", 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.715 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6815", 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6809", 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.716 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6811", 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43474 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6803/43474", 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6807/43474", 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6815/43474", 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6811/43474", 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.717 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "a47a3673-abbd-4827-966a-d68b17f3a18f", 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 0.854644775390625, 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:22.718 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 172, 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6800", 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6802", 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.719 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6804", 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6806", 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.720 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6812", 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6814", 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.721 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6808", 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6810", 2024-04-24T18:10:22.722 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43473 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6802/43473", 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6806/43473", 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6814/43473", 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6810/43473", 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-04-24T18:10:22.723 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "de3910ae-cc81-45d4-8639-cf2415330e64", 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 166, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.724 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6824", 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6825", 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6826", 2024-04-24T18:10:22.725 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6827", 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.726 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6830", 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6831", 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6828", 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.727 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6829", 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43481 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6825/43481", 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6827/43481", 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6831/43481", 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6829/43481", 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.728 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0", 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 172, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.729 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6816", 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6817", 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.730 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6818", 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6819", 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.731 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6822", 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6823", 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.732 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6820", 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.49:6821", 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43475 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.733 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.49:6817/43475", 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.49:6819/43475", 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.49:6823/43475", 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.49:6821/43475", 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "47f1fd84-e601-4e48-8058-bfa9296a9f90", 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.734 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 169, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6800", 2024-04-24T18:10:22.735 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6801", 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6802", 2024-04-24T18:10:22.736 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6803", 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6806", 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.737 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6807", 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6804", 2024-04-24T18:10:22.738 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6805", 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43499 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6801/43499", 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6803/43499", 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6807/43499", 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6805/43499", 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.739 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "1796d83b-591c-4c9d-b376-e36a2416a17e", 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:22.740 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 171, 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6808", 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6809", 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.741 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6810", 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6811", 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.742 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6814", 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6815", 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.743 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6812", 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6813", 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43500 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.744 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6809/43500", 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6811/43500", 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6815/43500", 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6813/43500", 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.745 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "5f203bac-3db4-4129-96cc-63dc06ee5ffa", 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 13, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 67, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 95, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 169, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 69, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.746 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6824", 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6825", 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.747 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6826", 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6827", 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.748 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6830", 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6831", 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.749 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6828", 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6829", 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 58122 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.750 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6825/58122", 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6827/58122", 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6831/58122", 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6829/58122", 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.751 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "uuid": "51061805-fbe5-4e80-be6b-dfc37cec0221", 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "up": 1, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "in": 1, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "weight": 1, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_affinity": 1, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_begin": 0, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "last_clean_end": 0, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "up_from": 13, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "up_thru": 169, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "down_at": 0, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "lost_at": 0, 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addrs": { 2024-04-24T18:10:22.752 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6816", 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6817", 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.753 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addrs": { 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6818", 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6819", 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.754 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addrs": { 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6822", 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6823", 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.755 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addrs": { 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "addrvec": [ 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v2", 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6820", 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "type": "v1", 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "addr": "172.21.15.78:6821", 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: "nonce": 43501 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.756 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "public_addr": "172.21.15.78:6817/43501", 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "cluster_addr": "172.21.15.78:6819/43501", 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_back_addr": "172.21.15.78:6823/43501", 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "heartbeat_front_addr": "172.21.15.78:6821/43501", 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "state": [ 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "exists", 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "up" 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "osd_xinfo": [ 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 0, 2024-04-24T18:10:22.757 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.248445+0000", 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 1, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.758 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.246633+0000", 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 2, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.176542+0000", 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:22.759 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 3, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.192111+0000", 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 4, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.760 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.151845+0000", 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 5, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.187089+0000", 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.761 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 6, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.255937+0000", 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 69 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "osd": 7, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "down_stamp": "0.000000", 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_probability": 0, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "laggy_interval": 0, 2024-04-24T18:10:22.762 INFO:teuthology.orchestra.run.smithi049.stdout: "features": 4540138320759226367, 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: "old_weight": 0, 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: "last_purged_snaps_scrub": "2024-04-24T18:07:15.160381+0000", 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: "dead_epoch": 0 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap": [ 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: "pgid": "4.1f", 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: "osds": [ 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: 5, 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: 2 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.763 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_upmap_items": [], 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "pg_temp": [], 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "primary_temp": [], 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "blocklist": {}, 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "range_blocklist": {}, 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "erasure_code_profiles": { 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "default": { 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "crush-failure-domain": "osd", 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "k": "2", 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "m": "1", 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "plugin": "jerasure", 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "technique": "reed_sol_van" 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: "removed_snaps_queue": [ 2024-04-24T18:10:22.764 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "pool": 4, 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "snaps": [ 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "begin": 4, 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "length": 4 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: ], 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "new_removed_snaps": [], 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "new_purged_snaps": [], 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "crush_node_flags": {}, 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "device_class_flags": {}, 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode": { 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_enabled": false, 2024-04-24T18:10:22.765 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_bucket_count": 0, 2024-04-24T18:10:22.766 INFO:teuthology.orchestra.run.smithi049.stdout: "degraded_stretch_mode": 0, 2024-04-24T18:10:22.766 INFO:teuthology.orchestra.run.smithi049.stdout: "recovering_stretch_mode": 0, 2024-04-24T18:10:22.766 INFO:teuthology.orchestra.run.smithi049.stdout: "stretch_mode_bucket": 0 2024-04-24T18:10:22.766 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.766 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:22.768 INFO:tasks.thrashosds.thrasher:j is {'epoch': 174, 'fsid': '5dae32d6-9101-4f7b-a6ef-ca5186fee71b', 'created': '2024-04-24T18:07:07.515859+0000', 'modified': '2024-04-24T18:10:22.323824+0000', 'last_up_change': '2024-04-24T18:08:52.865354+0000', 'last_in_change': '2024-04-24T18:10:17.313880+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 5, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 4, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'quincy', 'pools': [{'pool': 1, 'pool_name': 'rbd', 'create_time': '2024-04-24T18:07:18.850062+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [3], 'tier_of': -1, 'read_tier': 3, 'write_tier': 3, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 2, 'pool_name': '.mgr', 'create_time': '2024-04-24T18:07:19.353026+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 3, 'pool_name': 'cache', 'create_time': '2024-04-24T18:07:25.800822+0000', 'flags': 9, 'flags_names': 'hashpspool,incomplete_clones', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 4, 'pg_placement_num': 4, 'pg_placement_num_target': 4, 'pg_num_target': 4, 'pg_num_pending': 4, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '25', 'last_force_op_resend': '21', 'last_force_op_resend_prenautilus': '21', 'last_force_op_resend_preluminous': '21', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 19, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': 1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'writeback', 'target_max_bytes': 0, 'target_max_objects': 250, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'bloom', 'false_positive_probability': 0.05, 'target_size': 0, 'seed': 0}, 'hit_set_period': 60, 'hit_set_count': 8, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000, 1000000], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 4, 'pool_name': 'pool_client.0', 'create_time': '2024-04-24T18:07:35.513947+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 107, 'snap_epoch': 172, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}], 'osds': [{'osd': 0, 'uuid': '24da2d2a-953e-4440-9f6d-fb19cbec9ca1', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 173, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6801', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6803', 'nonce': 43474}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6805', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6807', 'nonce': 43474}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6813', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6815', 'nonce': 43474}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6809', 'nonce': 43474}, {'type': 'v1', 'addr': '172.21.15.49:6811', 'nonce': 43474}]}, 'public_addr': '172.21.15.49:6803/43474', 'cluster_addr': '172.21.15.49:6807/43474', 'heartbeat_back_addr': '172.21.15.49:6815/43474', 'heartbeat_front_addr': '172.21.15.49:6811/43474', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': 'a47a3673-abbd-4827-966a-d68b17f3a18f', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 0.854644775390625, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6800', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6802', 'nonce': 43473}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6804', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6806', 'nonce': 43473}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6812', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6814', 'nonce': 43473}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6808', 'nonce': 43473}, {'type': 'v1', 'addr': '172.21.15.49:6810', 'nonce': 43473}]}, 'public_addr': '172.21.15.49:6802/43473', 'cluster_addr': '172.21.15.49:6806/43473', 'heartbeat_back_addr': '172.21.15.49:6814/43473', 'heartbeat_front_addr': '172.21.15.49:6810/43473', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': 'de3910ae-cc81-45d4-8639-cf2415330e64', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 166, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6824', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6825', 'nonce': 43481}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6826', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6827', 'nonce': 43481}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6830', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6831', 'nonce': 43481}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6828', 'nonce': 43481}, {'type': 'v1', 'addr': '172.21.15.49:6829', 'nonce': 43481}]}, 'public_addr': '172.21.15.49:6825/43481', 'cluster_addr': '172.21.15.49:6827/43481', 'heartbeat_back_addr': '172.21.15.49:6831/43481', 'heartbeat_front_addr': '172.21.15.49:6829/43481', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 172, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6816', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6817', 'nonce': 43475}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6818', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6819', 'nonce': 43475}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6822', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6823', 'nonce': 43475}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.49:6820', 'nonce': 43475}, {'type': 'v1', 'addr': '172.21.15.49:6821', 'nonce': 43475}]}, 'public_addr': '172.21.15.49:6817/43475', 'cluster_addr': '172.21.15.49:6819/43475', 'heartbeat_back_addr': '172.21.15.49:6823/43475', 'heartbeat_front_addr': '172.21.15.49:6821/43475', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '47f1fd84-e601-4e48-8058-bfa9296a9f90', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 169, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6800', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6801', 'nonce': 43499}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6802', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6803', 'nonce': 43499}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6806', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6807', 'nonce': 43499}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6804', 'nonce': 43499}, {'type': 'v1', 'addr': '172.21.15.78:6805', 'nonce': 43499}]}, 'public_addr': '172.21.15.78:6801/43499', 'cluster_addr': '172.21.15.78:6803/43499', 'heartbeat_back_addr': '172.21.15.78:6807/43499', 'heartbeat_front_addr': '172.21.15.78:6805/43499', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '1796d83b-591c-4c9d-b376-e36a2416a17e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 171, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6808', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6809', 'nonce': 43500}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6810', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6811', 'nonce': 43500}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6814', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6815', 'nonce': 43500}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6812', 'nonce': 43500}, {'type': 'v1', 'addr': '172.21.15.78:6813', 'nonce': 43500}]}, 'public_addr': '172.21.15.78:6809/43500', 'cluster_addr': '172.21.15.78:6811/43500', 'heartbeat_back_addr': '172.21.15.78:6815/43500', 'heartbeat_front_addr': '172.21.15.78:6813/43500', 'state': ['exists', 'up']}, {'osd': 6, 'uuid': '5f203bac-3db4-4129-96cc-63dc06ee5ffa', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 13, 'last_clean_end': 67, 'up_from': 95, 'up_thru': 169, 'down_at': 69, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6824', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6825', 'nonce': 58122}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6826', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6827', 'nonce': 58122}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6830', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6831', 'nonce': 58122}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6828', 'nonce': 58122}, {'type': 'v1', 'addr': '172.21.15.78:6829', 'nonce': 58122}]}, 'public_addr': '172.21.15.78:6825/58122', 'cluster_addr': '172.21.15.78:6827/58122', 'heartbeat_back_addr': '172.21.15.78:6831/58122', 'heartbeat_front_addr': '172.21.15.78:6829/58122', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '51061805-fbe5-4e80-be6b-dfc37cec0221', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 169, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6816', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6817', 'nonce': 43501}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6818', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6819', 'nonce': 43501}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6822', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6823', 'nonce': 43501}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.78:6820', 'nonce': 43501}, {'type': 'v1', 'addr': '172.21.15.78:6821', 'nonce': 43501}]}, 'public_addr': '172.21.15.78:6817/43501', 'cluster_addr': '172.21.15.78:6819/43501', 'heartbeat_back_addr': '172.21.15.78:6823/43501', 'heartbeat_front_addr': '172.21.15.78:6821/43501', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.248445+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.246633+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.176542+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.192111+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.151845+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.187089+0000', 'dead_epoch': 0}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.255937+0000', 'dead_epoch': 69}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540138320759226367, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-04-24T18:07:15.160381+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '4.1f', 'osds': [5, 2]}], 'pg_upmap_items': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [{'pool': 4, 'snaps': [{'begin': 4, 'length': 4}]}], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-04-24T18:10:22.768 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 4.1f 2024-04-24T18:10:22.768 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 4.1f 2024-04-24T18:10:22.769 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:22.769 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:22.769 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:22.769 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:22.769 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:22.770 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-04-24T18:10:22.951 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:22.951 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:22.951 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:22.951 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:22.951 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58457 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 72704~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.821766239, 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059456347, 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58457 2024-04-24T18:10:22.952 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.953 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136973+0000", 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136975+0000", 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9379999999999999e-06 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136980+0000", 2024-04-24T18:10:22.954 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5400000000000003e-06 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136984+0000", 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.484e-06 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.193654+0000", 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056670127000000001 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.955 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196289+0000", 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.0026347900000000001 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196429+0000", 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00014078800000000001 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58458 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 76800~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:22.956 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.821751699, 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059714274999999997, 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58458 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.957 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136988+0000", 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136987+0000", 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136989+0000", 2024-04-24T18:10:22.958 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.029e-06 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136995+0000", 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6099999999999997e-06 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.136998+0000", 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5260000000000002e-06 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196459+0000", 2024-04-24T18:10:22.959 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059460644999999999 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196548+0000", 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9123999999999996e-05 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196702+0000", 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000153664 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.960 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58459 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 80896~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.821736848, 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946300000000001, 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58459 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.961 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137002+0000", 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.962 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137004+0000", 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1950000000000002e-06 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137010+0000", 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5720000000000002e-06 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137014+0000", 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.777e-06 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.963 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196731+0000", 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059717422999999999 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196829+0000", 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.7429000000000005e-05 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196949+0000", 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000120234 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.964 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58460 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 84992~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82172169899999, 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060175815000000001, 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58460 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.965 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137018+0000", 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137017+0000", 2024-04-24T18:10:22.966 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137019+0000", 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.154e-06 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137025+0000", 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.57e-06 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137028+0000", 2024-04-24T18:10:22.967 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4520000000000002e-06 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.196975+0000", 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.059946079999999999 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197058+0000", 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2974999999999998e-05 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197193+0000", 2024-04-24T18:10:22.968 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000135909 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58461 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 89088~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82170691300001, 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060400584, 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58461 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.969 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137032+0000", 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.970 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137034+0000", 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.9599999999999999e-06 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137040+0000", 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5010000000000004e-06 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137043+0000", 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.444e-06 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.971 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197222+0000", 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060178769999999999 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197307+0000", 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5273999999999997e-05 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197433+0000", 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012596100000000001 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.972 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58462 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 93184~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.821692324, 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060621250000000002, 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58462 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:22.973 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137047+0000", 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137053+0000", 2024-04-24T18:10:22.974 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8799999999999996e-06 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137059+0000", 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.353e-06 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137063+0000", 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9559999999999999e-06 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197461+0000", 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060397821999999997 2024-04-24T18:10:22.975 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197545+0000", 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.4691999999999995e-05 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197668+0000", 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012286600000000001 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.976 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58463 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 97280~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.821669499, 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060832455000000001, 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58463 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.977 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137070+0000", 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137069+0000", 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137072+0000", 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.52e-06 2024-04-24T18:10:22.978 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137078+0000", 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0859999999999998e-06 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137082+0000", 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.619e-06 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197696+0000", 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060613887999999998 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.979 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197810+0000", 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000114917 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197902+0000", 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1815000000000001e-05 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58464 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 101376~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:22.980 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82165384499999, 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061002309999999997, 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58464 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.981 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137085+0000", 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137087+0000", 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0059999999999999e-06 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.982 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137093+0000", 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.8150000000000002e-06 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137096+0000", 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4879999999999999e-06 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.197930+0000", 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060833498999999999 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.983 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198010+0000", 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0050999999999996e-05 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198088+0000", 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7757999999999997e-05 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58465 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 105472~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82163872300001, 2024-04-24T18:10:22.984 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061598891000000003, 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58465 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.985 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137101+0000", 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137100+0000", 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137103+0000", 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3159999999999999e-06 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.986 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137109+0000", 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0440000000000001e-06 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137112+0000", 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.738e-06 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198112+0000", 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.060999312999999999 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198194+0000", 2024-04-24T18:10:22.987 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2007000000000003e-05 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198699+0000", 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00050583099999999999 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58466 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 109568~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82162290700001, 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061855385999999998, 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.988 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58466 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.989 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137116+0000", 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137118+0000", 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.091e-06 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137124+0000", 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.609e-06 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.990 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137127+0000", 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.6969999999999999e-06 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198728+0000", 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061600395000000002 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198843+0000", 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000115306 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.991 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198972+0000", 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00012861400000000001 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58467 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 113664~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82160790500001, 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062189333999999999, 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58467 2024-04-24T18:10:22.992 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137131+0000", 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.993 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137133+0000", 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0109999999999999e-06 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137139+0000", 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6570000000000002e-06 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137142+0000", 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.596e-06 2024-04-24T18:10:22.994 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199001+0000", 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.061858165 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199090+0000", 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9316999999999999e-05 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199321+0000", 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00023090400000000001 2024-04-24T18:10:22.995 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58468 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 117760~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.821593041, 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062412440999999999, 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58468 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.996 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137146+0000", 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.997 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137148+0000", 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.995e-06 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137159+0000", 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.084e-05 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137163+0000", 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.0040000000000001e-06 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.998 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199351+0000", 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062188078000000001 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199433+0000", 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2182999999999996e-05 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199559+0000", 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000125661 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:22.999 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58469 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 121856~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82157192400001, 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.063272674000000001, 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58469 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.000 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137167+0000", 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.001 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137169+0000", 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137175+0000", 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9340000000000003e-06 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137179+0000", 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4790000000000001e-06 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.002 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199587+0000", 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062408574000000001 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199664+0000", 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7171999999999999e-05 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200440+0000", 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00077563700000000003 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.003 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58470 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 125952~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82155669400001, 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062812193000000002, 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58470 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.004 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137183+0000", 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137182+0000", 2024-04-24T18:10:23.005 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137184+0000", 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0260000000000002e-06 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137190+0000", 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6690000000000003e-06 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.006 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137193+0000", 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4659999999999999e-06 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199820+0000", 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062626608 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199902+0000", 2024-04-24T18:10:23.007 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1910999999999996e-05 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199995+0000", 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2843000000000005e-05 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58471 4.1c 4:3c946da3:::rbd_data.1174a7b75344.0000000000000006:17 [read 130048~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.008 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.82154195699999, 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062991447000000006, 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58471 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.009 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137197+0000", 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137199+0000", 2024-04-24T18:10:23.010 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.0099999999999998e-06 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137205+0000", 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.5339999999999999e-06 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.137208+0000", 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.2629999999999999e-06 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200019+0000", 2024-04-24T18:10:23.011 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.062810868000000006 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200099+0000", 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.064e-05 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9468000000000005e-05 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.012 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58642 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 131072~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.816440747, 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058296832999999999, 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58642 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.013 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142299+0000", 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142297+0000", 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.014 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142305+0000", 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2409999999999995e-06 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142319+0000", 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.3412e-05 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.015 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142326+0000", 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5089999999999996e-06 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200212+0000", 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.05788592 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200300+0000", 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8084999999999994e-05 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.016 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200595+0000", 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000295202 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58644 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 135168~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.816404901, 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058468749, 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.017 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58644 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.018 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142334+0000", 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.019 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.020 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142337+0000", 2024-04-24T18:10:23.020 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.4249999999999998e-06 2024-04-24T18:10:23.020 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.020 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.020 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142344+0000", 2024-04-24T18:10:23.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9650000000000002e-06 2024-04-24T18:10:23.021 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.021 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.021 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.021 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142349+0000", 2024-04-24T18:10:23.021 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.7190000000000001e-06 2024-04-24T18:10:23.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.022 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.022 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.022 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200631+0000", 2024-04-24T18:10:23.022 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058281873999999997 2024-04-24T18:10:23.022 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.023 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200710+0000", 2024-04-24T18:10:23.023 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9333999999999996e-05 2024-04-24T18:10:23.023 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.023 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.023 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200803+0000", 2024-04-24T18:10:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2974999999999997e-05 2024-04-24T18:10:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.024 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58646 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 139264~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.81638489400001, 2024-04-24T18:10:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058641086000000002, 2024-04-24T18:10:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.025 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.026 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.026 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58646 2024-04-24T18:10:23.026 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.026 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.027 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:23.027 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.027 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.027 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.027 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:23.028 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.028 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.028 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.028 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.028 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142354+0000", 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142359+0000", 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2730000000000002e-06 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.029 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.030 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142366+0000", 2024-04-24T18:10:23.030 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7360000000000004e-06 2024-04-24T18:10:23.030 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.030 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.030 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.142370+0000", 2024-04-24T18:10:23.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4599999999999996e-06 2024-04-24T18:10:23.031 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.031 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.031 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.031 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200830+0000", 2024-04-24T18:10:23.031 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058459516000000003 2024-04-24T18:10:23.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.032 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.032 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.032 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200905+0000", 2024-04-24T18:10:23.032 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4754999999999998e-05 2024-04-24T18:10:23.032 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.033 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.033 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.033 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200996+0000", 2024-04-24T18:10:23.033 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.0785999999999994e-05 2024-04-24T18:10:23.033 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.033 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.034 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.034 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.034 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.034 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58647 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 143360~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.034 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:23.034 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.814563703, 2024-04-24T18:10:23.035 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057037266000000003, 2024-04-24T18:10:23.035 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.035 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.035 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.036 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58647 2024-04-24T18:10:23.036 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.036 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.036 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.036 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.036 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:23.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.037 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.037 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.037 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.037 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144176+0000", 2024-04-24T18:10:23.037 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144175+0000", 2024-04-24T18:10:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.999999 2024-04-24T18:10:23.038 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.039 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144180+0000", 2024-04-24T18:10:23.039 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.3229999999999997e-06 2024-04-24T18:10:23.039 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.039 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.039 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144190+0000", 2024-04-24T18:10:23.040 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.3610000000000007e-06 2024-04-24T18:10:23.040 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.040 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.040 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.040 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144195+0000", 2024-04-24T18:10:23.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.942e-06 2024-04-24T18:10:23.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.041 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.041 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.041 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201022+0000", 2024-04-24T18:10:23.041 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826983999999997 2024-04-24T18:10:23.041 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201112+0000", 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9435e-05 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201213+0000", 2024-04-24T18:10:23.042 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000101012 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58649 4.1c 4:385c9b02:::rbd_data.1174a7b75344.0000000000000005:17 [read 147456~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 130.81453842299999, 2024-04-24T18:10:23.043 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057222708999999997, 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58649 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.044 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:23.045 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.045 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.045 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.045 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.045 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144201+0000", 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144200+0000", 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144203+0000", 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.7750000000000001e-06 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.046 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144209+0000", 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0979999999999999e-06 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.144214+0000", 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.4109999999999998e-06 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201241+0000", 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057027605000000002 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.047 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201330+0000", 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8152999999999994e-05 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201424+0000", 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.4086999999999996e-05 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.048 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.049 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:23.050 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-04-24T18:10:23.180 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:23.180 INFO:teuthology.orchestra.run.smithi049.stdout: "num_ops": 0 2024-04-24T18:10:23.180 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:23.190 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-04-24T18:10:23.319 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [], 2024-04-24T18:10:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "complaint_time": 30, 2024-04-24T18:10:23.319 INFO:teuthology.orchestra.run.smithi049.stdout: "num_blocked_ops": 0 2024-04-24T18:10:23.319 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:23.329 INFO:teuthology.orchestra.run.smithi049.stderr:clear 4.1f pg_upmap mapping 2024-04-24T18:10:23.332 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-04-24T18:10:23.463 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "size": 20, 2024-04-24T18:10:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 600, 2024-04-24T18:10:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "ops": [ 2024-04-24T18:10:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.463 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:58823 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 126976~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.29222720600001, 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058875552999999997, 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 58823 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.464 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178480+0000", 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178479+0000", 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.465 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178482+0000", 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3949999999999999e-06 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178489+0000", 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.7150000000000001e-06 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.178493+0000", 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2010000000000001e-06 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.466 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237192+0000", 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.058699032999999998 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237275+0000", 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2906999999999998e-05 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.237355+0000", 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.0672000000000003e-05 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.467 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59283 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 47104~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27258807999999, 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056223091000000003, 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59283 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.468 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198119+0000", 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.469 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198121+0000", 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3939999999999999e-06 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198127+0000", 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2790000000000004e-06 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198132+0000", 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2139999999999998e-06 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.470 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254171+0000", 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056039110000000003 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254250+0000", 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9284999999999997e-05 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254342+0000", 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.2168000000000003e-05 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.471 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59284 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 51200~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.272556358, 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056377797, 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59284 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.472 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198151+0000", 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198150+0000", 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.473 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198153+0000", 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.9399999999999998e-06 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198165+0000", 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1871999999999999e-05 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198170+0000", 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.459e-06 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.474 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254370+0000", 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056200700999999999 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254442+0000", 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2131000000000004e-05 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254528+0000", 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6145999999999998e-05 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.475 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59285 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 55296~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27253294100001, 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056541102000000003, 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59285 2024-04-24T18:10:23.476 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198174+0000", 2024-04-24T18:10:23.477 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198181+0000", 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9399999999999996e-06 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198187+0000", 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2609999999999999e-06 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.478 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198191+0000", 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1980000000000003e-06 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254557+0000", 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056366325000000002 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254639+0000", 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.1476000000000004e-05 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.479 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254715+0000", 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.6289000000000001e-05 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59286 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 59392~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.272502868, 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056694199000000001, 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.480 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59286 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198204+0000", 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198195+0000", 2024-04-24T18:10:23.481 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999909 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198207+0000", 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 1.1399999999999999e-05 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198213+0000", 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1190000000000002e-06 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.198218+0000", 2024-04-24T18:10:23.482 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.0599999999999998e-06 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254740+0000", 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056522216 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254819+0000", 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9442000000000003e-05 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.254898+0000", 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.9023000000000006e-05 2024-04-24T18:10:23.483 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59290 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 75776~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27145772099999, 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056354433000000002, 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59290 2024-04-24T18:10:23.484 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199249+0000", 2024-04-24T18:10:23.485 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199251+0000", 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.1430000000000001e-06 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199257+0000", 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9170000000000003e-06 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199262+0000", 2024-04-24T18:10:23.486 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.9400000000000001e-06 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255455+0000", 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056193003999999998 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255529+0000", 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3868000000000005e-05 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255604+0000", 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.4881000000000004e-05 2024-04-24T18:10:23.487 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59291 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 79872~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27144049, 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056514271999999997, 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59291 2024-04-24T18:10:23.488 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199267+0000", 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199266+0000", 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.489 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199269+0000", 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.401e-06 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199275+0000", 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.8510000000000001e-06 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199280+0000", 2024-04-24T18:10:23.490 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1080000000000003e-06 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255629+0000", 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056349630999999997 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255702+0000", 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2782999999999998e-05 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255781+0000", 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8877999999999999e-05 2024-04-24T18:10:23.491 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59292 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 83968~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.271423492, 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056674064000000003, 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59292 2024-04-24T18:10:23.492 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199284+0000", 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199283+0000", 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.493 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199286+0000", 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2960000000000001e-06 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199291+0000", 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.6579999999999999e-06 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199296+0000", 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.2989999999999996e-06 2024-04-24T18:10:23.494 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255807+0000", 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056510126000000001 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255882+0000", 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5544999999999996e-05 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255958+0000", 2024-04-24T18:10:23.495 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5489999999999997e-05 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59293 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 88064~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.271404811, 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056831645, 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59293 2024-04-24T18:10:23.496 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199302+0000", 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.497 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199304+0000", 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4389999999999999e-06 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199310+0000", 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.0589999999999999e-06 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.498 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199314+0000", 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8600000000000003e-06 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.255983+0000", 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056668598000000001 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256056+0000", 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3620000000000003e-05 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.499 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256134+0000", 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7426e-05 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59294 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 92160~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27138734299999, 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056995275999999997, 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.500 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59294 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199320+0000", 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.501 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199319+0000", 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199322+0000", 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5129999999999999e-06 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199328+0000", 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9410000000000004e-06 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.502 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199332+0000", 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9689999999999996e-06 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256166+0000", 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056833957999999997 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256237+0000", 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1806999999999999e-05 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.503 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256315+0000", 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7546999999999995e-05 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59295 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 96256~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.271371219, 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057153925000000001, 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.504 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59295 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199336+0000", 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.505 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199335+0000", 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199338+0000", 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.3089999999999998e-06 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199344+0000", 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9020000000000004e-06 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.506 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199347+0000", 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.467e-06 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256340+0000", 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056993007999999998 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256411+0000", 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.1233000000000001e-05 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256490+0000", 2024-04-24T18:10:23.507 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.8356000000000007e-05 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59296 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 100352~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27134011300001, 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057305648000000001, 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59296 2024-04-24T18:10:23.508 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199367+0000", 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.509 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199370+0000", 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.033e-06 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199377+0000", 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3490000000000003e-06 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199382+0000", 2024-04-24T18:10:23.510 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.6979999999999997e-06 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256515+0000", 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057133382000000003 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256590+0000", 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.5043999999999995e-05 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256673+0000", 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2571000000000006e-05 2024-04-24T18:10:23.511 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59297 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 104448~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27132091199999, 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057458120000000001, 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59297 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.512 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199386+0000", 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.513 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199392+0000", 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6939999999999998e-06 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199399+0000", 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3280000000000002e-06 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199403+0000", 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:10:23.514 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256698+0000", 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057295190000000003 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256771+0000", 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.2792000000000004e-05 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256844+0000", 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.3590999999999994e-05 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.515 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59298 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 108544~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27129987000001, 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057572950999999997, 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59298 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.516 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199407+0000", 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.517 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199409+0000", 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.6000000000000001e-06 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199416+0000", 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.3600000000000001e-06 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199419+0000", 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.5130000000000001e-06 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.518 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256860+0000", 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057440535000000001 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256903+0000", 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2818000000000002e-05 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.256980+0000", 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 7.7514000000000004e-05 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.519 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59299 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 112640~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.270544922, 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057029219999999999, 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59299 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.520 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200162+0000", 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.199423+0000", 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9992609 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.521 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200166+0000", 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00074291400000000001 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200175+0000", 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.1959999999999997e-06 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200181+0000", 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.4870000000000002e-06 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.522 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257007+0000", 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056826557999999999 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257095+0000", 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.7244999999999996e-05 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257191+0000", 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 9.6720999999999998e-05 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.523 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59300 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 116736~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27051842, 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204415000000002, 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59300 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.524 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200189+0000", 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200188+0000", 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.525 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200191+0000", 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200197+0000", 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6309999999999997e-06 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200202+0000", 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.1529999999999999e-06 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.526 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257221+0000", 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057019546999999997 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257307+0000", 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.5772000000000005e-05 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257393+0000", 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.6187999999999995e-05 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.527 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59301 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 120832~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.27050088999999, 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057387913999999998, 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59301 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.528 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200206+0000", 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.529 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200208+0000", 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.4109999999999999e-06 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200214+0000", 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 5.9200000000000001e-06 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.200218+0000", 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.552e-06 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.530 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257422+0000", 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.057204603 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257505+0000", 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2756999999999994e-05 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.257594+0000", 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.9064e-05 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.531 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59303 4.19 4:9ba93581:::rbd_data.1174a7b75344.0000000000000008:17 [read 129024~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.26903773000001, 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.066022754000000003, 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59303 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.532 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201669+0000", 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201672+0000", 2024-04-24T18:10:23.533 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.604e-06 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201678+0000", 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.2480000000000001e-06 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.201682+0000", 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.9609999999999999e-06 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267188+0000", 2024-04-24T18:10:23.534 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.065506033000000005 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267254+0000", 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.6161999999999994e-05 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.267692+0000", 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000438171 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.535 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59368 4.11 4:8db2b4ed:::rbd_data.11747e3f2a60.0000000000000000:15 [read 1570816~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.257199082, 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056279836999999999, 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59368 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:23.536 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213508+0000", 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213510+0000", 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.2819999999999999e-06 2024-04-24T18:10:23.537 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213516+0000", 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.1589999999999998e-06 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213520+0000", 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 3.8410000000000002e-06 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269602+0000", 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056081837000000002 2024-04-24T18:10:23.538 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269684+0000", 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.2105000000000001e-05 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269788+0000", 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.000104008 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.539 INFO:teuthology.orchestra.run.smithi049.stdout: "description": "osd_op(client.4468.0:59401 4.11 4:8b0435a3:::rbd_data.1174a7b75344.0000000000000002:17 [read 134144~1024] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e60)", 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "initiated_at": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "age": 131.257182739, 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056480619000000003, 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "type_data": { 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "flag_point": "started", 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client_info": { 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client": "client.4468", 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "tid": 59401 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "events": [ 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "initiated", 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:23.540 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "throttled", 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "header_read", 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213524+0000", 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "all_read", 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213526+0000", 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 2.368e-06 2024-04-24T18:10:23.541 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "dispatched", 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213533+0000", 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 6.9539999999999998e-06 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.213538+0000", 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 4.2579999999999997e-06 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "reached_pg", 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269816+0000", 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.056278697000000003 2024-04-24T18:10:23.542 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "started", 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.269905+0000", 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 8.8438999999999998e-05 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: }, 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: { 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: "event": "done", 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: "time": "2024-04-24T18:08:12.270005+0000", 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: "duration": 0.00010027900000000001 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: } 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout: ] 2024-04-24T18:10:23.543 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:23.544 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-04-24T18:10:23.682 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:23.682 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:23.682 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:23.683 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:23.692 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-04-24T18:10:23.815 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:23.815 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:23.815 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:23.815 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:23.815 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:23.825 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-04-24T18:10:23.951 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:23.951 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11800 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2875392~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61972943699999, 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042494045000000001, 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11800 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.952 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339027+0000", 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339026+0000", 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339029+0000", 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339035+0000", 2024-04-24T18:10:23.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9850000000000003e-06 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339039+0000", 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.788e-06 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381377+0000", 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042338886999999999 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381456+0000", 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8448999999999994e-05 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381521+0000", 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4949000000000004e-05 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11801 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2883584~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.619713531, 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042690302999999999, 2024-04-24T18:10:23.956 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11801 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339043+0000", 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339042+0000", 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339045+0000", 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.356e-06 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339051+0000", 2024-04-24T18:10:23.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8470000000000001e-06 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339054+0000", 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.76e-06 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381560+0000", 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042506059999999998 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381653+0000", 2024-04-24T18:10:23.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3166999999999998e-05 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381733+0000", 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9499999999999994e-05 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11802 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2893824~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61966156400001, 2024-04-24T18:10:23.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042827048, 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11802 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339095+0000", 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339058+0000", 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999633 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339097+0000", 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9243999999999999e-05 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339105+0000", 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7540000000000006e-06 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.339110+0000", 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0769999999999998e-06 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.964 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381767+0000", 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042656645 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381847+0000", 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0484999999999999e-05 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.381922+0000", 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4350000000000005e-05 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.965 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11806 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2930688~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61873808999999, 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042470866000000003, 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.966 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11806 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340018+0000", 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340020+0000", 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3630000000000001e-06 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340026+0000", 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0299999999999999e-06 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340031+0000", 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4989999999999996e-06 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382329+0000", 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042298615999999997 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382417+0000", 2024-04-24T18:10:23.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7590000000000007e-05 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382489+0000", 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2136000000000002e-05 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11807 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2938880~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:10:23.970 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61872145800001, 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042649990999999998, 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11807 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340035+0000", 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340034+0000", 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340037+0000", 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2390000000000001e-06 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340043+0000", 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5390000000000004e-06 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340047+0000", 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5700000000000001e-06 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382523+0000", 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042476119999999999 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382610+0000", 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7323000000000004e-05 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.382685+0000", 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4559000000000002e-05 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.974 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11810 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2975744~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.618019855, 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042538873999999997, 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11810 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340736+0000", 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340738+0000", 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5730000000000002e-06 2024-04-24T18:10:23.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340745+0000", 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0730000000000001e-06 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340748+0000", 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.839e-06 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383122+0000", 2024-04-24T18:10:23.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042373369000000001 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383204+0000", 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2149000000000002e-05 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383275+0000", 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1339000000000003e-05 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.978 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11811 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 2983936~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61800335300001, 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042681157999999997, 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11811 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.979 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340753+0000", 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340752+0000", 2024-04-24T18:10:23.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340755+0000", 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4739999999999999e-06 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340761+0000", 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9009999999999999e-06 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.340765+0000", 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.811e-06 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383314+0000", 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042549558000000001 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383393+0000", 2024-04-24T18:10:23.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8413999999999998e-05 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.383434+0000", 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1436000000000001e-05 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11815 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3020800~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61710048699999, 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042495165000000001, 2024-04-24T18:10:23.983 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11815 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341656+0000", 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341657+0000", 2024-04-24T18:10:23.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3120000000000001e-06 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341660+0000", 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9679999999999998e-06 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341662+0000", 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9759999999999998e-06 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384042+0000", 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042380653999999997 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384110+0000", 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7189999999999999e-05 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384151+0000", 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1279000000000002e-05 2024-04-24T18:10:23.987 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11816 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3039232~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61709246199999, 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042658886, 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11816 2024-04-24T18:10:23.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341664+0000", 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341666+0000", 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.813e-06 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341669+0000", 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0170000000000001e-06 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.341671+0000", 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.857e-06 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384177+0000", 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042505347999999998 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.991 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384246+0000", 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9550999999999999e-05 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.384323+0000", 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6507000000000005e-05 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11821 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3084288~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:10:23.992 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61602832, 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042434151000000003, 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11821 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342728+0000", 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342730+0000", 2024-04-24T18:10:23.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5160000000000001e-06 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342736+0000", 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9070000000000004e-06 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342740+0000", 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6509999999999999e-06 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385063+0000", 2024-04-24T18:10:23.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042322961999999999 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385122+0000", 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9784999999999998e-05 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385162+0000", 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9719999999999999e-05 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:23.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:11822 4.1a 4:5a0dcf4a:::rbd_data.11749ffb1524.0000000000000037:9 [read 3094528~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e38)", 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 154.61601235099999, 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042544472, 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 11822 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:23.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342744+0000", 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:23.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342746+0000", 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.148e-06 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342751+0000", 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7200000000000003e-06 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.342755+0000", 2024-04-24T18:10:23.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6770000000000001e-06 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385186+0000", 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042430742 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385229+0000", 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3582999999999999e-05 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:49.385288+0000", 2024-04-24T18:10:24.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8946000000000001e-05 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:92891 4.d 4:b04b1ecb:::rbd_data.1174787cbd24.0000000000000054:head [call rbd.sparse_copyup in=16b,set-alloc-hint object_size 262144 write_size 262144,delete] snapc 21=[21] RETRY=1 ondisk+retry+write+known_if_redirected+supports_pool_eio e73)", 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 117.555813912, 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.047959374999999999, 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.001 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 92891 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402942+0000", 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402944+0000", 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.058e-06 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402947+0000", 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0369999999999999e-06 2024-04-24T18:10:24.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.402950+0000", 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8590000000000001e-06 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432775+0000", 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.029825059000000001 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.432850+0000", 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4536000000000006e-05 2024-04-24T18:10:24.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 2", 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.433071+0000", 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00022133400000000001 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.436539+0000", 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034681400000000002 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450814+0000", 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.014275354000000001 2024-04-24T18:10:24.005 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450864+0000", 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9895999999999998e-05 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:26.450902+0000", 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7339000000000003e-05 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136786 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 222208~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 107.38678987999999, 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042630663999999999, 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136786 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571966+0000", 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571968+0000", 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2630000000000002e-06 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571975+0000", 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.55e-06 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571978+0000", 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.568e-06 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614414+0000", 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042436011000000003 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614494+0000", 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9457999999999997e-05 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614597+0000", 2024-04-24T18:10:24.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010314000000000001 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136787 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 229376~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 107.38677364599999, 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042816286000000002, 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.011 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136787 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571983+0000", 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571982+0000", 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571984+0000", 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2910000000000002e-06 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571990+0000", 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6439999999999997e-06 2024-04-24T18:10:24.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571994+0000", 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6100000000000002e-06 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614634+0000", 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042640431999999999 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614706+0000", 2024-04-24T18:10:24.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2187999999999994e-05 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614799+0000", 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2601000000000003e-05 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136788 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 236544~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 107.386758426, 2024-04-24T18:10:24.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043015194, 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136788 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571998+0000", 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.571997+0000", 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572003+0000", 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9699999999999996e-06 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572009+0000", 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8429999999999998e-06 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572013+0000", 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8060000000000001e-06 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614831+0000", 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042818002000000001 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.614912+0000", 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1199000000000006e-05 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615013+0000", 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000100691 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136789 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 243712~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 107.386730847, 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043164237000000001, 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.020 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136789 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572025+0000", 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572028+0000", 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9500000000000001e-06 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572034+0000", 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6880000000000002e-06 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572039+0000", 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4390000000000002e-06 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615039+0000", 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042999902 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615102+0000", 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3105e-05 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615190+0000", 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.7709999999999999e-05 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.024 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136790 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 250880~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:24.025 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:10:24.025 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 107.386712546, 2024-04-24T18:10:24.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043350289, 2024-04-24T18:10:24.025 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.025 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.025 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136790 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572044+0000", 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572043+0000", 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572046+0000", 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6979999999999998e-06 2024-04-24T18:10:24.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572052+0000", 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8919999999999997e-06 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572055+0000", 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4759999999999998e-06 2024-04-24T18:10:24.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615226+0000", 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043171036000000003 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615313+0000", 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6372999999999994e-05 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615394+0000", 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1229000000000004e-05 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:136791 4.13 4:ce73e09c:::rbd_data.11743af029fe.000000000000000a:27 [read 258048~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e81)", 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 107.386696247, 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043560326000000003, 2024-04-24T18:10:24.030 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 136791 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:10:24.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572060+0000", 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572062+0000", 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3690000000000001e-06 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572068+0000", 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5260000000000001e-06 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.572071+0000", 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5540000000000002e-06 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615433+0000", 2024-04-24T18:10:24.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.043361591999999997 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615511+0000", 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8776999999999995e-05 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:36.615620+0000", 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000108862 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.034 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:172968 4.18 4:1d66092b:::rbd_header.1174290f8ab0:head [watch reconnect cookie 139789846434464 gen 2] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e96)", 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 90.078293627999997, 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.098757236, 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 172968 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880463+0000", 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880461+0000", 2024-04-24T18:10:24.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999981 2024-04-24T18:10:24.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880467+0000", 2024-04-24T18:10:24.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.297e-06 2024-04-24T18:10:24.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880478+0000", 2024-04-24T18:10:24.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1141999999999999e-05 2024-04-24T18:10:24.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880487+0000", 2024-04-24T18:10:24.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4579999999999993e-06 2024-04-24T18:10:24.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880537+0000", 2024-04-24T18:10:24.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0637000000000003e-05 2024-04-24T18:10:24.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for peered", 2024-04-24T18:10:24.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:53.880550+0000", 2024-04-24T18:10:24.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2322e-05 2024-04-24T18:10:24.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977072+0000", 2024-04-24T18:10:24.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0965217469999999 2024-04-24T18:10:24.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.977178+0000", 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000106041 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:54.979220+0000", 2024-04-24T18:10:24.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020422610000000001 2024-04-24T18:10:24.043 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.043 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.043 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.043 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:188905 4.1b 4:df50bad4:::rbd_data.11747889028a.0000000000000010:32 [read 305152~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:24.043 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:10:24.044 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 84.729020523000003, 2024-04-24T18:10:24.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042499417999999997, 2024-04-24T18:10:24.044 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.044 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.044 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.044 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.045 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.045 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 188905 2024-04-24T18:10:24.045 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.045 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.045 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.045 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.045 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229736+0000", 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.046 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229735+0000", 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229737+0000", 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3510000000000001e-06 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229740+0000", 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.932e-06 2024-04-24T18:10:24.047 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.229742+0000", 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.883e-06 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272108+0000", 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.042366055999999999 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272148+0000", 2024-04-24T18:10:24.048 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0726999999999998e-05 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.272235+0000", 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.6679000000000001e-05 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.049 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:24.051 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-04-24T18:10:24.174 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:24.175 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:24.175 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:24.175 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:24.184 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-04-24T18:10:24.307 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:24.308 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:24.308 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:24.308 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:24.308 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:24.318 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-04-24T18:10:24.341 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-04-24T18:10:24.444 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:24.444 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:24.444 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:24.444 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:24.444 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.444 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122752 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 471040~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.444 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.08758206500001, 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049956188999999998, 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122752 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.445 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363746+0000", 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363745+0000", 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.446 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363747+0000", 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2470000000000001e-06 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363749+0000", 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8100000000000002e-06 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363752+0000", 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1660000000000001e-06 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.447 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413585+0000", 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049833536999999997 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413636+0000", 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1001000000000002e-05 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.448 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413702+0000", 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5646000000000001e-05 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122753 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 458752~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.087573936, 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164089000000002, 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.449 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122753 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.450 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363754+0000", 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363755+0000", 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.15e-06 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363758+0000", 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8559999999999998e-06 2024-04-24T18:10:24.451 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363760+0000", 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.108e-06 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413728+0000", 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.049967878 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413797+0000", 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9638e-05 2024-04-24T18:10:24.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413918+0000", 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120672 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122759 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 503808~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.08756591300001, 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050290174, 2024-04-24T18:10:24.453 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122759 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363762+0000", 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363763+0000", 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.296e-06 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363766+0000", 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8049999999999998e-06 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363768+0000", 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.911e-06 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413943+0000", 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050175341999999998 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.413988+0000", 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.456e-05 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414052+0000", 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4472000000000002e-05 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122760 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 491520~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:24.457 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.087557, 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050507477000000002, 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122760 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:24.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363771+0000", 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363772+0000", 2024-04-24T18:10:24.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.488e-06 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363775+0000", 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.835e-06 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363777+0000", 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4770000000000002e-06 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414079+0000", 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050301541999999998 2024-04-24T18:10:24.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414158+0000", 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9175e-05 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414278+0000", 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120172 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.461 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122767 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 407552~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.087548181, 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050636159, 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122767 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363780+0000", 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363779+0000", 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363781+0000", 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5179999999999999e-06 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363784+0000", 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8909999999999999e-06 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363786+0000", 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.052e-06 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414304+0000", 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050518672000000001 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414349+0000", 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5005e-05 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414416+0000", 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6241000000000005e-05 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.465 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122768 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 395264~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.087539049, 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05085895, 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122768 2024-04-24T18:10:24.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363789+0000", 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363788+0000", 2024-04-24T18:10:24.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363790+0000", 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2190000000000001e-06 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363793+0000", 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.9620000000000002e-06 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363795+0000", 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7889999999999999e-06 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414442+0000", 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050646587999999999 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414522+0000", 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9808999999999994e-05 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414648+0000", 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012579600000000001 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122775 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 440320~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.08671879000001, 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050164085999999997, 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.470 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122775 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364609+0000", 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.363797+0000", 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9991884 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364613+0000", 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081525499999999997 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364621+0000", 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4880000000000007e-06 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364625+0000", 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.715e-06 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414660+0000", 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050035108000000002 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414707+0000", 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.7278999999999999e-05 2024-04-24T18:10:24.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414773+0000", 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5846000000000006e-05 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122776 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 428032~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086691591, 2024-04-24T18:10:24.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050359896000000001, 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122776 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364636+0000", 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364638+0000", 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6589999999999999e-06 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364644+0000", 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9039999999999997e-06 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364649+0000", 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.8670000000000002e-06 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414800+0000", 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050151451 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414878+0000", 2024-04-24T18:10:24.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7945e-05 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.414996+0000", 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000117594 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122783 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 473088~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086674121, 2024-04-24T18:10:24.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050477611999999998, 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122783 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364654+0000", 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364653+0000", 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364655+0000", 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1919999999999999e-06 2024-04-24T18:10:24.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364661+0000", 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.429e-06 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364665+0000", 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.861e-06 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415021+0000", 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.05035618 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415066+0000", 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4823000000000001e-05 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415131+0000", 2024-04-24T18:10:24.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5483999999999998e-05 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122784 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 460800~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086658781, 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050667859000000003, 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122784 2024-04-24T18:10:24.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364669+0000", 2024-04-24T18:10:24.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364678+0000", 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0100000000000001e-06 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364684+0000", 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3820000000000001e-06 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364688+0000", 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4050000000000002e-06 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415159+0000", 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050470229999999998 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415224+0000", 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5012999999999996e-05 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415337+0000", 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000113193 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122791 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 505856~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086634703, 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050778130999999997, 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122791 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364693+0000", 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364695+0000", 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.176e-06 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364701+0000", 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7069999999999997e-06 2024-04-24T18:10:24.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364704+0000", 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9020000000000001e-06 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415361+0000", 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050656594999999999 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415406+0000", 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4920999999999999e-05 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415471+0000", 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5188e-05 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122792 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 493568~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086619138, 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050968934, 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.492 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122792 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364709+0000", 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364708+0000", 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364710+0000", 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1210000000000001e-06 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364716+0000", 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.215e-06 2024-04-24T18:10:24.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364719+0000", 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8290000000000001e-06 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415493+0000", 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050773658999999999 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415572+0000", 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8870999999999998e-05 2024-04-24T18:10:24.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415678+0000", 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00010559499999999999 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122798 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 397312~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086604177, 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051083982, 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.496 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122798 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364724+0000", 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364723+0000", 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364725+0000", 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0839999999999999e-06 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364731+0000", 2024-04-24T18:10:24.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2129999999999996e-06 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364736+0000", 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.78e-06 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415697+0000", 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.050960234 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415742+0000", 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5899000000000002e-05 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415808+0000", 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5121000000000002e-05 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122802 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 430080~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.500 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.08658723800001, 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051192724000000002, 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122802 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364740+0000", 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364742+0000", 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.024e-06 2024-04-24T18:10:24.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364748+0000", 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6489999999999996e-06 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364751+0000", 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.5080000000000001e-06 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415826+0000", 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051074373999999999 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415867+0000", 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0936000000000002e-05 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415933+0000", 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6562000000000004e-05 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.504 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122806 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 462848~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086570833, 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051307414000000003, 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122806 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364757+0000", 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364756+0000", 2024-04-24T18:10:24.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364759+0000", 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364764+0000", 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6749999999999999e-06 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364769+0000", 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.622e-06 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415952+0000", 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051182658999999998 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.415997+0000", 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5528000000000002e-05 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416064+0000", 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7210000000000002e-05 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.508 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122810 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 495616~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086554394, 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051521895999999998, 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122810 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364773+0000", 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364775+0000", 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.154e-06 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364780+0000", 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4140000000000002e-06 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364784+0000", 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.726e-06 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416093+0000", 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051309187999999999 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416173+0000", 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9583e-05 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416295+0000", 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012227000000000001 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.512 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122814 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 399360~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086539554, 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051642475, 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122814 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.513 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364788+0000", 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364790+0000", 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.029e-06 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364796+0000", 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.02e-06 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364800+0000", 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0330000000000002e-06 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416320+0000", 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051520059 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416364+0000", 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3831000000000001e-05 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416431+0000", 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6823999999999996e-05 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122818 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 432128~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.517 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.08652313, 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851691999999998, 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122818 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364805+0000", 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364804+0000", 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364807+0000", 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2689999999999998e-06 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364812+0000", 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1139999999999997e-06 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364816+0000", 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2610000000000004e-06 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416456+0000", 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051639603999999999 2024-04-24T18:10:24.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416533+0000", 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7317000000000005e-05 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416656+0000", 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012342599999999999 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.521 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122822 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 464896~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.086505915, 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051970211000000002, 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122822 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364822+0000", 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364821+0000", 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364824+0000", 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.277e-06 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364829+0000", 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.972e-06 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364831+0000", 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1679999999999998e-06 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416682+0000", 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051851062000000003 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416726+0000", 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4144000000000003e-05 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416792+0000", 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.5980000000000002e-05 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.525 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:122826 4.1f 4:fa13714d:::rbd_data.117498712027.0000000000000005:1f [read 497664~2048] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e76)", 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 114.08648845, 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.052171846000000001, 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 122826 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364839+0000", 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364841+0000", 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.649e-06 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364845+0000", 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.121e-06 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.364847+0000", 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3740000000000001e-06 2024-04-24T18:10:24.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416813+0000", 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.051965692000000001 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.416890+0000", 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7426e-05 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:30.417011+0000", 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120828 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.529 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.530 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.530 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.530 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:24.531 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-04-24T18:10:24.656 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:24.657 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:24.657 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:24.657 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:24.666 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-04-24T18:10:24.790 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:24.791 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:24.791 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:24.791 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:24.791 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:24.800 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-04-24T18:10:24.926 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:24.926 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:24.926 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:24.927 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:24.927 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.927 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:113 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.927 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:24.927 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 184.316213925, 2024-04-24T18:10:24.927 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011036706, 2024-04-24T18:10:24.928 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.928 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.928 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.928 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.928 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.928 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 113 2024-04-24T18:10:24.929 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.929 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.929 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.929 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.929 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:24.929 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.930 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.930 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.930 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616423+0000", 2024-04-24T18:10:24.930 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.930 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.931 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.931 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616422+0000", 2024-04-24T18:10:24.931 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.931 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.931 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.932 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616427+0000", 2024-04-24T18:10:24.932 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5140000000000003e-06 2024-04-24T18:10:24.932 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.932 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.932 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616433+0000", 2024-04-24T18:10:24.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4339999999999997e-06 2024-04-24T18:10:24.933 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.933 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.933 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.933 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616438+0000", 2024-04-24T18:10:24.933 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2919999999999997e-06 2024-04-24T18:10:24.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.934 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.934 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621249+0000", 2024-04-24T18:10:24.934 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0048111300000000003 2024-04-24T18:10:24.934 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.934 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621329+0000", 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0174999999999997e-05 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621464+0000", 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00013542599999999999 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.935 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623464+0000", 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019994430000000001 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627376+0000", 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0039119189999999998 2024-04-24T18:10:24.936 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627422+0000", 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6230000000000003e-05 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.937 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627460+0000", 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7716999999999998e-05 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:114 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 184.31618911499999, 2024-04-24T18:10:24.938 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011096323, 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 114 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.939 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616448+0000", 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616447+0000", 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616451+0000", 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2810000000000002e-06 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.940 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616458+0000", 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.1600000000000003e-06 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616460+0000", 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6989999999999999e-06 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621754+0000", 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0052933399999999997 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.941 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621804+0000", 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0250000000000002e-05 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.621917+0000", 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00011292499999999999 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623505+0000", 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0015880309999999999 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.942 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627469+0000", 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.003964581 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627510+0000", 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0238000000000003e-05 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627544+0000", 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4375000000000002e-05 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.943 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.945 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.945 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.945 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:115 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.945 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:24.945 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 184.316169003, 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011163809, 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 115 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.946 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616468+0000", 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.947 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616523+0000", 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5708000000000003e-05 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616530+0000", 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.9619999999999996e-06 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616535+0000", 2024-04-24T18:10:24.948 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3440000000000003e-06 2024-04-24T18:10:24.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.949 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.949 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.949 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622318+0000", 2024-04-24T18:10:24.949 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0057833679999999997 2024-04-24T18:10:24.949 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622391+0000", 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.3574999999999999e-05 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622518+0000", 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012694900000000001 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.950 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623550+0000", 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010318580000000001 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627552+0000", 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040012770000000001 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627589+0000", 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7357999999999997e-05 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.951 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627632+0000", 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2657999999999997e-05 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:116 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 184.31609834, 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011177878, 2024-04-24T18:10:24.952 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 116 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.953 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616538+0000", 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.954 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616541+0000", 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.593e-06 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616545+0000", 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8750000000000002e-06 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616547+0000", 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2029999999999999e-06 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.955 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.622916+0000", 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.006368884 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623009+0000", 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3812000000000002e-05 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623130+0000", 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000120281 2024-04-24T18:10:24.956 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624061+0000", 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00093122099999999998 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627640+0000", 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035793800000000001 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627682+0000", 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1264999999999999e-05 2024-04-24T18:10:24.957 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627716+0000", 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4666999999999999e-05 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:117 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 184.31608656700001, 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.011245342, 2024-04-24T18:10:24.958 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 117 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.959 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616550+0000", 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616552+0000", 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5009999999999999e-06 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.960 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616556+0000", 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.083e-06 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.616558+0000", 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3640000000000002e-06 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623568+0000", 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.007009777 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.961 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623623+0000", 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4818000000000003e-05 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.623702+0000", 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.9498000000000004e-05 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.624517+0000", 2024-04-24T18:10:24.962 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00081529900000000002 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627724+0000", 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0032062060000000001 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627764+0000", 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.0414000000000003e-05 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:20.627796+0000", 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1596e-05 2024-04-24T18:10:24.963 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:137 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 183.895764738, 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092491199999999996, 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.964 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 137 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:24.965 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036872+0000", 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036902+0000", 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0741999999999998e-05 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.966 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036909+0000", 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4810000000000003e-06 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036914+0000", 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1610000000000002e-06 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040420+0000", 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0035060930000000001 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.967 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040502+0000", 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.1593000000000003e-05 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.040635+0000", 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000132774 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042051+0000", 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001416661 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046064+0000", 2024-04-24T18:10:24.968 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0040129520000000002 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046101+0000", 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6850999999999997e-05 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046121+0000", 2024-04-24T18:10:24.969 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0122000000000001e-05 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:138 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 183.89571755599999, 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092434370000000002, 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.970 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 138 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.971 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036919+0000", 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.972 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036921+0000", 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.475e-06 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036926+0000", 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4229999999999998e-06 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036929+0000", 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1530000000000001e-06 2024-04-24T18:10:24.973 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041115+0000", 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0041858520000000003 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041157+0000", 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2040999999999999e-05 2024-04-24T18:10:24.974 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041232+0000", 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5178000000000004e-05 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.975 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042697+0000", 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001464829 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046125+0000", 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034284010000000002 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.976 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046146+0000", 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0774000000000001e-05 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046163+0000", 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6558999999999998e-05 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:139 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.977 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 183.895692435, 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092550289999999997, 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 139 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.978 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036944+0000", 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.979 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036948+0000", 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4379999999999997e-06 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036956+0000", 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.0940000000000008e-06 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036962+0000", 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.2449999999999998e-06 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.980 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041506+0000", 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0045444320000000002 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041584+0000", 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7547999999999997e-05 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.041712+0000", 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000128582 2024-04-24T18:10:24.981 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042740+0000", 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.001027535 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.982 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046166+0000", 2024-04-24T18:10:24.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0034259939999999999 2024-04-24T18:10:24.983 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.983 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.983 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.983 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046185+0000", 2024-04-24T18:10:24.983 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8947e-05 2024-04-24T18:10:24.984 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.984 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.984 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.984 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046199+0000", 2024-04-24T18:10:24.984 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4765e-05 2024-04-24T18:10:24.984 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:140 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 183.89564030099999, 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092425809999999997, 2024-04-24T18:10:24.985 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 140 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:24.986 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.036996+0000", 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037041+0000", 2024-04-24T18:10:24.987 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5491999999999997e-05 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037051+0000", 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.8490000000000001e-06 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037055+0000", 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7809999999999999e-06 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042080+0000", 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0050249500000000002 2024-04-24T18:10:24.988 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042157+0000", 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7058999999999995e-05 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042283+0000", 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00012606999999999999 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043358+0000", 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0010751770000000001 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.989 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046202+0000", 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00284424 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046221+0000", 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8377999999999999e-05 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046239+0000", 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8301999999999999e-05 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.990 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4194.0:141 2.0 2:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 183.89557683699999, 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092167700000000009, 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4194", 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.78:0/2512206168", 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 141 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.991 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037060+0000", 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.992 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037062+0000", 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8629999999999999e-06 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037066+0000", 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3830000000000001e-06 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.037068+0000", 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1339999999999998e-06 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.993 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042752+0000", 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0056841299999999999 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042796+0000", 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.4323999999999998e-05 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "waiting for subops from 1", 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.042872+0000", 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.5143999999999997e-05 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.994 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "sub_op_commit_rec", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.043763+0000", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00089114499999999998 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "op_commit", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046242+0000", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0024794550000000002 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046260+0000", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7725999999999999e-05 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:21.046277+0000", 2024-04-24T18:10:24.995 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.685e-05 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33445 4.15 e42/26 4:a92aadb4:::rbd_data.117432c990a0.000000000000002a:head v 42'52, mlcod=42'45)", 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 152.12207812, 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.010446462, 2024-04-24T18:10:24.996 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810559+0000", 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.997 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810558+0000", 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810563+0000", 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0289999999999996e-06 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810572+0000", 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.9209999999999998e-06 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.998 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.810576+0000", 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1509999999999997e-06 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811888+0000", 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0013114680000000001 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811908+0000", 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9979e-05 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:24.999 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820867+0000", 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089591740000000003 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.821005+0000", 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.000138331 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:33455 4.1e e42/26 4:786b9a36:::rbd_data.117432c990a0.0000000000000034:head v 42'404, mlcod=42'399)", 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:25.000 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 152.120972051, 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092535429999999995, 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811665+0000", 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.001 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811664+0000", 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811673+0000", 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.3009999999999995e-06 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811683+0000", 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4490000000000005e-06 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.002 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811689+0000", 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7409999999999998e-06 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811720+0000", 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1318000000000001e-05 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.811733+0000", 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3417999999999999e-05 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.003 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820884+0000", 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091504269999999992 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:07:52.820918+0000", 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4594000000000001e-05 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.004 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:199190 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'516, mlcod=102'514)", 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 84.903846388999995, 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0092843779999999994, 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.005 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028790+0000", 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.006 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028789+0000", 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028801+0000", 2024-04-24T18:10:25.007 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2085e-05 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028821+0000", 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0115e-05 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028830+0000", 2024-04-24T18:10:25.008 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5299999999999996e-06 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028886+0000", 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6162999999999997e-05 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.009 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.028910+0000", 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4374000000000001e-05 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038025+0000", 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0091144199999999998 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.010 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.038075+0000", 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9948999999999999e-05 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.011 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207242 4.17 e102/26 4:e811a87e:::rbd_object_map.11741dae6a9e:head v 102'552, mlcod=102'548)", 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 84.563383235000003, 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0086221579999999996, 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.012 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369254+0000", 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.013 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369252+0000", 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999986 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369297+0000", 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5291999999999999e-05 2024-04-24T18:10:25.014 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369318+0000", 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.0951e-05 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369327+0000", 2024-04-24T18:10:25.015 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0399999999999998e-06 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369379+0000", 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.151e-05 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.016 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369401+0000", 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2609e-05 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377835+0000", 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084330150000000003 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.017 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377876+0000", 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1190999999999999e-05 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:207243 4.17 e102/26 4:ef021b8a:::rbd_data.11741dae6a9e.0000000000000011:head v 102'553, mlcod=102'548)", 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 84.563293540999993, 2024-04-24T18:10:25.018 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0085634349999999995, 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.019 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369343+0000", 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369859+0000", 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00051668799999999996 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369879+0000", 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9201000000000001e-05 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.020 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369885+0000", 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6660000000000002e-06 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.369981+0000", 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5518000000000003e-05 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.370000+0000", 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9219e-05 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.021 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377878+0000", 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078782049999999992 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:09:00.377907+0000", 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8484e-05 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.022 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(osd.5.0:1562 4.8 e160/26 4:12f07558:::rbd_data.117432c990a0.0000000000000005:a v 160'686, mlcod=156'683)", 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:10:08.474534+0000", 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 16.458102529000001, 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0090488800000000005, 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474534+0000", 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474534+0000", 2024-04-24T18:10:25.023 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474534+0000", 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474539+0000", 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6029999999999998e-06 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.024 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474547+0000", 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.303e-06 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474552+0000", 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.1039999999999998e-06 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.025 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.475480+0000", 2024-04-24T18:10:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00092805999999999997 2024-04-24T18:10:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.026 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.475505+0000", 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5349000000000001e-05 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.483547+0000", 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0080414470000000002 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.483583+0000", 2024-04-24T18:10:25.027 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6242999999999998e-05 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(osd.5.0:1564 4.8 e160/26 4:16f6b57a:::rbd_data.117432c990a0.0000000000000030:a v 160'688, mlcod=156'683)", 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:10:08.474589+0000", 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 16.458048228999999, 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0090244290000000005, 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.028 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474589+0000", 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474589+0000", 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474588+0000", 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.029 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474592+0000", 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6320000000000001e-06 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474597+0000", 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0679999999999996e-06 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.474606+0000", 2024-04-24T18:10:25.030 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0869999999999996e-06 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.476590+0000", 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0019839979999999998 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.476614+0000", 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.4168999999999998e-05 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.031 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.483587+0000", 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0069730670000000003 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.483613+0000", 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5644e-05 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(osd.3.0:5392 4.0 e160/26 4:03227239:::rbd_data.117432c990a0.000000000000017e:a v 160'807, mlcod=155'802)", 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:10:08.478660+0000", 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 16.453976541999999, 2024-04-24T18:10:25.032 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0099577829999999996, 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.478660+0000", 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.478660+0000", 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.033 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.478659+0000", 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.478667+0000", 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.6059999999999997e-06 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.478678+0000", 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.076e-05 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.034 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.478684+0000", 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2910000000000004e-06 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.480704+0000", 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0020195679999999998 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.480723+0000", 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.9225e-05 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.035 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.488575+0000", 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0078521800000000003 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:08.488618+0000", 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3019999999999998e-05 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:231969 4.3 e165/164 4:c2114a66:::rbd_data.1174b1b97c76.0000000000000009:head v 165'942, mlcod=41'45)", 2024-04-24T18:10:25.036 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:10:13.413451+0000", 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 11.519185606000001, 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0090760790000000008, 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413451+0000", 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413451+0000", 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.037 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413451+0000", 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413455+0000", 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.155e-06 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413461+0000", 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.0340000000000002e-06 2024-04-24T18:10:25.038 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413465+0000", 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.4960000000000001e-06 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413539+0000", 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.4008e-05 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.413560+0000", 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1619000000000001e-05 2024-04-24T18:10:25.039 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.422491+0000", 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0089305159999999995 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.422527+0000", 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6551999999999998e-05 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.040 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_repop(client.4468.0:231970 4.3 e165/164 4:c28f2cd5:::rbd_data.1174b1b97c76.000000000000000b:head v 165'943, mlcod=41'45)", 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:10:13.414090+0000", 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 11.518546382, 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0084637429999999993, 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "commit sent; apply or cleanup", 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414090+0000", 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414090+0000", 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.041 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414090+0000", 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414096+0000", 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.3609999999999997e-06 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414106+0000", 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0180000000000001e-05 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.042 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414112+0000", 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.6710000000000004e-06 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414150+0000", 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7750999999999998e-05 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.414171+0000", 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1226999999999999e-05 2024-04-24T18:10:25.043 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "commit_sent", 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.422530+0000", 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.0083594199999999994 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:10:13.422554+0000", 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3731e-05 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.044 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:25.046 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-04-24T18:10:25.170 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:25.170 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:25.170 INFO:teuthology.orchestra.run.smithi078.stdout: "num_ops": 0 2024-04-24T18:10:25.171 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:25.180 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-04-24T18:10:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:25.304 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [], 2024-04-24T18:10:25.304 INFO:teuthology.orchestra.run.smithi078.stdout: "complaint_time": 30, 2024-04-24T18:10:25.304 INFO:teuthology.orchestra.run.smithi078.stdout: "num_blocked_ops": 0 2024-04-24T18:10:25.304 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:25.314 DEBUG:teuthology.orchestra.run.smithi078:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-04-24T18:10:25.338 INFO:teuthology.orchestra.run.smithi049.stderr:noscrub is unset 2024-04-24T18:10:25.351 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-04-24T18:10:25.450 INFO:teuthology.orchestra.run.smithi078.stdout:{ 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "size": 20, 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 600, 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "ops": [ 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193833 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2560000~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.939425184000001, 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033994705, 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.451 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193833 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.452 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518759+0000", 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518766+0000", 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1740000000000003e-06 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518772+0000", 2024-04-24T18:10:25.453 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.178e-06 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518775+0000", 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.3289999999999998e-06 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552565+0000", 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033789784000000003 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552656+0000", 2024-04-24T18:10:25.454 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.0816000000000006e-05 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552754+0000", 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7769000000000005e-05 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193834 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2617344~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.939404249000006, 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034077348, 2024-04-24T18:10:25.455 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193834 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.456 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518780+0000", 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518779+0000", 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518782+0000", 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2230000000000001e-06 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.457 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518800+0000", 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8544999999999999e-05 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518805+0000", 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.561e-06 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552766+0000", 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033961065999999998 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.458 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552808+0000", 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2135999999999998e-05 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552857+0000", 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9317000000000002e-05 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:193835 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2629632~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:25.459 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.939374490999995, 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034267886999999997, 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 193835 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.460 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518810+0000", 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518809+0000", 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518812+0000", 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.6809999999999998e-06 2024-04-24T18:10:25.461 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518818+0000", 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2020000000000001e-06 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.518827+0000", 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.183e-06 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552889+0000", 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034061620000000001 2024-04-24T18:10:25.462 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.552978+0000", 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.8759999999999997e-05 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.553078+0000", 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.9938000000000004e-05 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.463 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194013 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2617344~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935869996999998, 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033797673, 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194013 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:25.464 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522314+0000", 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522323+0000", 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.2690000000000005e-06 2024-04-24T18:10:25.465 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522329+0000", 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2890000000000003e-06 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522334+0000", 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5469999999999998e-06 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.555973+0000", 2024-04-24T18:10:25.466 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033638821999999999 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556045+0000", 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2101000000000006e-05 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556112+0000", 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.7150999999999995e-05 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.467 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194014 4.5 4:a0106e5c:::rbd_data.1174290f8ab0.0000000000000006:34 [read 2629632~0] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935845662999995, 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033896114999999997, 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194014 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.468 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522339+0000", 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522338+0000", 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522341+0000", 2024-04-24T18:10:25.469 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3889999999999999e-06 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522347+0000", 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.6329999999999999e-06 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522350+0000", 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.2260000000000001e-06 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.470 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556144+0000", 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033794009999999999 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556193+0000", 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9070999999999998e-05 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556235+0000", 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1196999999999999e-05 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.471 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194073 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1773568~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935824073000006, 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034006096999999999, 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194073 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.472 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522360+0000", 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.473 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522362+0000", 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.3779999999999999e-06 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522368+0000", 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.8000000000000004e-06 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522372+0000", 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.7589999999999999e-06 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.474 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556261+0000", 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033889150999999999 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556304+0000", 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2746e-05 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556366+0000", 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2655000000000002e-05 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.475 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194074 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1830912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935807947000001, 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03416027, 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194074 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.476 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522376+0000", 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.477 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522378+0000", 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.2759999999999999e-06 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522384+0000", 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.5600000000000001e-06 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522387+0000", 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.1700000000000001e-06 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.478 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556398+0000", 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034011448 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556477+0000", 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.8819000000000005e-05 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556537+0000", 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9373000000000002e-05 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.479 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194075 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1843200~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935793054000001, 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034288482000000002, 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194075 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.480 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522391+0000", 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.481 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522393+0000", 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.1500000000000002e-06 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522398+0000", 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.4229999999999996e-06 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522402+0000", 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6650000000000001e-06 2024-04-24T18:10:25.482 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556570+0000", 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034167686000000003 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556620+0000", 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.0763999999999997e-05 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556680+0000", 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9168e-05 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.483 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194076 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1912832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935538288000004, 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034250926000000001, 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194076 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.484 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522646+0000", 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522406+0000", 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9997602 2024-04-24T18:10:25.485 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522650+0000", 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.00024406499999999999 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522661+0000", 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.0735e-05 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522667+0000", 2024-04-24T18:10:25.486 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.2829999999999998e-06 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556719+0000", 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034051396999999997 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556801+0000", 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2346000000000001e-05 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556897+0000", 2024-04-24T18:10:25.487 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.5917999999999999e-05 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194077 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 1982464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935510492999995, 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034406256000000003, 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.488 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194077 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522674+0000", 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.489 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522673+0000", 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522676+0000", 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.7539999999999998e-06 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522684+0000", 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.2050000000000002e-06 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.490 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522700+0000", 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.5549999999999999e-05 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556923+0000", 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034223477000000002 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.556996+0000", 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.2539999999999993e-05 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.491 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557080+0000", 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4159000000000005e-05 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194078 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2052096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935478227999994, 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034555871000000002, 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.492 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194078 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522706+0000", 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.493 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522705+0000", 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.999999 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522709+0000", 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.585e-06 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522717+0000", 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.4619999999999996e-06 2024-04-24T18:10:25.494 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522722+0000", 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.5990000000000004e-06 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557119+0000", 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034397625000000001 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557205+0000", 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5266000000000007e-05 2024-04-24T18:10:25.495 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557262+0000", 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.7213999999999998e-05 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194079 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2064384~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.935445303999998, 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034739542999999998, 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.496 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194079 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.497 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522739+0000", 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522741+0000", 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8090000000000001e-06 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522747+0000", 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.9259999999999997e-06 2024-04-24T18:10:25.498 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.522750+0000", 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.818e-06 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557295+0000", 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.03454496 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557381+0000", 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 8.5894000000000001e-05 2024-04-24T18:10:25.499 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.557478+0000", 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.7523999999999997e-05 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194085 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2342912~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933530204999997, 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033933133999999997, 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.500 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194085 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.501 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524654+0000", 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524655+0000", 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4169999999999999e-06 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524659+0000", 2024-04-24T18:10:25.502 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8920000000000002e-06 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524662+0000", 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5150000000000001e-06 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558455+0000", 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033793686000000003 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558533+0000", 2024-04-24T18:10:25.503 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7955999999999997e-05 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558587+0000", 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 5.3896e-05 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194086 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2412544~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933520197000007, 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034104519999999999, 2024-04-24T18:10:25.504 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194086 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.505 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524664+0000", 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524665+0000", 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.3370000000000001e-06 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.506 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524668+0000", 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.0400000000000001e-06 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524675+0000", 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.1169999999999998e-06 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558627+0000", 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.033951426999999999 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.507 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558691+0000", 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 6.4010999999999994e-05 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558769+0000", 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 7.7824000000000006e-05 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194087 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2424832~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933506750000006, 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034214107000000001, 2024-04-24T18:10:25.508 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194087 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.509 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524677+0000", 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524679+0000", 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4279999999999999e-06 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.510 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524682+0000", 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.999e-06 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524684+0000", 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.368e-06 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558796+0000", 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034112405999999998 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.511 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558845+0000", 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.9004e-05 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558892+0000", 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6134999999999998e-05 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194088 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2482176~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933498116999999, 2024-04-24T18:10:25.512 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034301085000000002, 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194088 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.513 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524686+0000", 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524687+0000", 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2020000000000001e-06 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.514 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524690+0000", 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8779999999999998e-06 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524692+0000", 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.5440000000000001e-06 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558906+0000", 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034213973000000002 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.515 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558945+0000", 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8232000000000003e-05 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.558987+0000", 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.2477e-05 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194089 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2494464~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:25.516 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933489620000003, 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034395917999999998, 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194089 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.517 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524695+0000", 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524694+0000", 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524696+0000", 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.1829999999999999e-06 2024-04-24T18:10:25.518 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524698+0000", 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.824e-06 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524700+0000", 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.7829999999999999e-06 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559002+0000", 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034302115000000001 2024-04-24T18:10:25.519 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559044+0000", 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1774999999999999e-05 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559090+0000", 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6443999999999998e-05 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.520 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194090 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2551808~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933481983999997, 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034488782000000003, 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194090 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.521 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524702+0000", 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524703+0000", 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.2440000000000001e-06 2024-04-24T18:10:25.522 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524706+0000", 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 2.8260000000000001e-06 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524708+0000", 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.6929999999999999e-06 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559104+0000", 2024-04-24T18:10:25.523 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034396719999999999 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559144+0000", 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.9857999999999997e-05 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559191+0000", 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.6655e-05 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.524 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194091 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2564096~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933474329999996, 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034576260999999997, 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194091 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:25.525 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524710+0000", 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967296 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524719+0000", 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 9.4159999999999993e-06 2024-04-24T18:10:25.526 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524722+0000", 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.18e-06 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524724+0000", 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.894e-06 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559206+0000", 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034481570000000003 2024-04-24T18:10:25.527 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559245+0000", 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8982e-05 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559286+0000", 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.1433999999999997e-05 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "description": "osd_op(client.4468.0:194092 4.5 4:a38f931f:::rbd_data.1174290f8ab0.000000000000000a:34 [read 2633728~4096] snapc 0=[] RETRY=2 ondisk+retry+read+known_if_redirected+supports_pool_eio e101)", 2024-04-24T18:10:25.528 INFO:teuthology.orchestra.run.smithi078.stdout: "initiated_at": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "age": 85.933457779999998, 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034654889000000001, 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "type_data": { 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "flag_point": "started", 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "client_info": { 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "client": "client.4468", 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "client_addr": "172.21.15.49:0/1131867777", 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "tid": 194092 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "events": [ 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "initiated", 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.529 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "throttled", 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "header_read", 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524726+0000", 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4294967295.9999995 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "all_read", 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524728+0000", 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.4869999999999999e-06 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.530 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "dispatched", 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524731+0000", 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.6909999999999999e-06 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "queued_for_pg", 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.524733+0000", 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 1.8330000000000001e-06 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "reached_pg", 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559299+0000", 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 0.034566426999999997 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.531 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "started", 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559338+0000", 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 3.8331999999999998e-05 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: }, 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: { 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "event": "done", 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "time": "2024-04-24T18:08:59.559381+0000", 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: "duration": 4.3494000000000001e-05 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: } 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout: ] 2024-04-24T18:10:25.532 INFO:teuthology.orchestra.run.smithi078.stdout:} 2024-04-24T18:10:26.343 INFO:teuthology.orchestra.run.smithi049.stderr:nodeep-scrub is unset 2024-04-24T18:10:28.343 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-04-24T18:10:28.492 INFO:teuthology.orchestra.run.smithi049.stdout:mon.a: {} 2024-04-24T18:10:28.502 INFO:teuthology.orchestra.run.smithi049.stdout:mon.b: {} 2024-04-24T18:10:28.512 INFO:teuthology.orchestra.run.smithi049.stdout:mon.c: {} 2024-04-24T18:10:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T18:10:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T18:10:28.513 INFO:teuthology.orchestra.run.smithi049.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-04-24T18:10:28.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-04-24T18:10:29.346 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.0 to 1 (10000) 2024-04-24T18:10:29.361 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-04-24T18:10:30.353 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-04-24T18:10:30.366 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-04-24T18:10:31.361 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.1 to 1 (10000) 2024-04-24T18:10:31.376 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-04-24T18:10:32.368 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-04-24T18:10:32.382 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-04-24T18:10:33.376 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.2 to 1 (10000) 2024-04-24T18:10:33.391 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-04-24T18:10:34.385 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-04-24T18:10:34.400 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-04-24T18:10:35.394 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.3 to 1 (10000) 2024-04-24T18:10:35.408 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-04-24T18:10:36.400 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-04-24T18:10:36.412 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-04-24T18:10:37.405 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.4 to 1 (10000) 2024-04-24T18:10:37.418 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-04-24T18:10:38.411 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-04-24T18:10:38.425 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-04-24T18:10:39.416 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.5 to 1 (10000) 2024-04-24T18:10:39.431 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-04-24T18:10:40.424 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-04-24T18:10:40.439 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-04-24T18:10:41.429 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.7 to 1 (10000) 2024-04-24T18:10:41.443 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-04-24T18:10:42.434 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-04-24T18:10:42.449 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-04-24T18:10:43.440 INFO:teuthology.orchestra.run.smithi049.stderr:reweighted osd.6 to 1 (10000) 2024-04-24T18:10:43.454 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-04-24T18:10:44.446 INFO:teuthology.orchestra.run.smithi049.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-04-24T18:10:44.460 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-04-24T18:10:44.460 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-04-24T18:10:44.460 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-04-24T18:10:44.460 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-04-24T18:10:44.460 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-24T18:10:44.460 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:10:44.801 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:44.802 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":195,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:10:44.445859+0000","last_up_change":"2024-04-24T18:08:52.865354+0000","last_in_change":"2024-04-24T18:10:17.313880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-24T18:07:18.850062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-24T18:07:19.353026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T18:07:25.800822+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":1,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T18:07:35.513947+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":107,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":95,"up_thru":178,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":58122},{"type":"v1","addr":"172.21.15.78:6825","nonce":58122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":58122},{"type":"v1","addr":"172.21.15.78:6827","nonce":58122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":58122},{"type":"v1","addr":"172.21.15.78:6831","nonce":58122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":58122},{"type":"v1","addr":"172.21.15.78:6829","nonce":58122}]},"public_addr":"172.21.15.78:6825/58122","cluster_addr":"172.21.15.78:6827/58122","heartbeat_back_addr":"172.21.15.78:6831/58122","heartbeat_front_addr":"172.21.15.78:6829/58122","state":["exists","up"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":69},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:10:44.816 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-24T18:10:44.816 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:10:45.148 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:45.148 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":195,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:10:44.445859+0000","last_up_change":"2024-04-24T18:08:52.865354+0000","last_in_change":"2024-04-24T18:10:17.313880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-24T18:07:18.850062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-24T18:07:19.353026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T18:07:25.800822+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":1,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T18:07:35.513947+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":107,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":95,"up_thru":178,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":58122},{"type":"v1","addr":"172.21.15.78:6825","nonce":58122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":58122},{"type":"v1","addr":"172.21.15.78:6827","nonce":58122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":58122},{"type":"v1","addr":"172.21.15.78:6831","nonce":58122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":58122},{"type":"v1","addr":"172.21.15.78:6829","nonce":58122}]},"public_addr":"172.21.15.78:6825/58122","cluster_addr":"172.21.15.78:6827/58122","heartbeat_back_addr":"172.21.15.78:6831/58122","heartbeat_front_addr":"172.21.15.78:6829/58122","state":["exists","up"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":69},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:10:45.162 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-24T18:10:45.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-24T18:10:45.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-24T18:10:45.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-24T18:10:45.164 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-04-24T18:10:45.164 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-04-24T18:10:45.164 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-04-24T18:10:45.165 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-04-24T18:10:45.459 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:45.459 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T18:10:45.460 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:45.461 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T18:10:45.469 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:45.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T18:10:45.470 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:45.471 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T18:10:45.475 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:45.475 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T18:10:45.485 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:45.485 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T18:10:45.487 INFO:teuthology.orchestra.run.smithi049.stdout:408021893144 2024-04-24T18:10:45.487 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T18:10:45.494 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:45.494 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T18:10:46.126 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:46.126 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:46.128 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:46.131 INFO:teuthology.orchestra.run.smithi049.stdout:55834574891 2024-04-24T18:10:46.143 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:46.152 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574892 for osd.3 2024-04-24T18:10:46.153 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574892 for osd.4 2024-04-24T18:10:46.153 INFO:teuthology.orchestra.run.smithi049.stdout:55834574891 2024-04-24T18:10:46.155 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574892 for osd.7 2024-04-24T18:10:46.162 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574892 got 55834574891 for osd.2 2024-04-24T18:10:46.162 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:46.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574892 for osd.0 2024-04-24T18:10:46.177 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574892 got 55834574891 for osd.1 2024-04-24T18:10:46.178 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574892 for osd.5 2024-04-24T18:10:46.188 INFO:teuthology.orchestra.run.smithi049.stdout:408021893143 2024-04-24T18:10:46.202 INFO:tasks.ceph.ceph_manager.ceph:need seq 408021893144 got 408021893143 for osd.6 2024-04-24T18:10:47.153 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-24T18:10:47.153 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-04-24T18:10:47.156 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-04-24T18:10:47.163 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-24T18:10:47.169 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-24T18:10:47.178 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-24T18:10:47.178 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-04-24T18:10:47.204 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-04-24T18:10:47.800 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:47.818 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:47.820 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574893 for osd.3 2024-04-24T18:10:47.820 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.843 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574893 for osd.4 2024-04-24T18:10:47.844 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.854 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:47.858 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:47.867 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:47.868 INFO:teuthology.orchestra.run.smithi049.stdout:55834574893 2024-04-24T18:10:47.870 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574893 for osd.0 2024-04-24T18:10:47.870 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.881 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574893 for osd.7 2024-04-24T18:10:47.882 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.883 INFO:teuthology.orchestra.run.smithi049.stdout:55834574892 2024-04-24T18:10:47.891 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574892 got 55834574892 for osd.1 2024-04-24T18:10:47.892 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.892 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574893 got 55834574893 for osd.5 2024-04-24T18:10:47.892 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.900 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574892 got 55834574892 for osd.2 2024-04-24T18:10:47.901 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.908 INFO:teuthology.orchestra.run.smithi049.stdout:408021893145 2024-04-24T18:10:47.922 INFO:tasks.ceph.ceph_manager.ceph:need seq 408021893144 got 408021893145 for osd.6 2024-04-24T18:10:47.922 DEBUG:teuthology.parallel:result is None 2024-04-24T18:10:47.922 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-04-24T18:10:47.922 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:10:48.292 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:48.293 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:10:48.304 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-24T18:10:47.337398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590640,"num_objects":1800,"num_object_clones":0,"num_object_copies":3600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29264,"num_write_kb":534157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26387,"ondisk_log_size":26387,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456004,"kb_used_data":18492,"kb_used_omap":43,"kb_used_meta":2437332,"kb_avail":752518716,"statfs":{"total":773094113280,"available":770579165184,"internally_reserved":0,"allocated":18935808,"data_stored":10771526,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2495828359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.905362"},"pg_stats":[{"pgid":"4.1f","version":"173'683","reported_seq":7230,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449490+0000","last_change":"2024-04-24T18:10:27.328087+0000","last_active":"2024-04-24T18:10:43.449490+0000","last_peered":"2024-04-24T18:10:43.449490+0000","last_clean":"2024-04-24T18:10:43.449490+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:10:43.449490+0000","last_undegraded":"2024-04-24T18:10:43.449490+0000","last_fullsized":"2024-04-24T18:10:43.449490+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:17.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"160'1339","reported_seq":7257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445000+0000","last_change":"2024-04-24T18:10:21.335419+0000","last_active":"2024-04-24T18:10:43.445000+0000","last_peered":"2024-04-24T18:10:43.445000+0000","last_clean":"2024-04-24T18:10:43.445000+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:43.445000+0000","last_undegraded":"2024-04-24T18:10:43.445000+0000","last_fullsized":"2024-04-24T18:10:43.445000+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1339,"log_dups_size":0,"ondisk_log_size":1339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:03.319026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"167'710","reported_seq":5997,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449714+0000","last_change":"2024-04-24T18:10:20.330289+0000","last_active":"2024-04-24T18:10:43.449714+0000","last_peered":"2024-04-24T18:10:43.449714+0000","last_clean":"2024-04-24T18:10:43.449714+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:10:43.449714+0000","last_undegraded":"2024-04-24T18:10:43.449714+0000","last_fullsized":"2024-04-24T18:10:43.449714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1079","reported_seq":10264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445678+0000","last_change":"2024-04-24T18:10:20.345289+0000","last_active":"2024-04-24T18:10:43.445678+0000","last_peered":"2024-04-24T18:10:43.445678+0000","last_clean":"2024-04-24T18:10:43.445678+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:43.445678+0000","last_undegraded":"2024-04-24T18:10:43.445678+0000","last_fullsized":"2024-04-24T18:10:43.445678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"171'535","reported_seq":11656,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449807+0000","last_change":"2024-04-24T18:10:20.332101+0000","last_active":"2024-04-24T18:10:43.449807+0000","last_peered":"2024-04-24T18:10:43.449807+0000","last_clean":"2024-04-24T18:10:43.449807+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:43.449807+0000","last_undegraded":"2024-04-24T18:10:43.449807+0000","last_fullsized":"2024-04-24T18:10:43.449807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"167'880","reported_seq":12337,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449756+0000","last_change":"2024-04-24T18:10:20.331679+0000","last_active":"2024-04-24T18:10:43.449756+0000","last_peered":"2024-04-24T18:10:43.449756+0000","last_clean":"2024-04-24T18:10:43.449756+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:43.449756+0000","last_undegraded":"2024-04-24T18:10:43.449756+0000","last_fullsized":"2024-04-24T18:10:43.449756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"167'726","reported_seq":9735,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449505+0000","last_change":"2024-04-24T18:10:20.329954+0000","last_active":"2024-04-24T18:10:43.449505+0000","last_peered":"2024-04-24T18:10:43.449505+0000","last_clean":"2024-04-24T18:10:43.449505+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:43.449505+0000","last_undegraded":"2024-04-24T18:10:43.449505+0000","last_fullsized":"2024-04-24T18:10:43.449505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"167'616","reported_seq":11663,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449727+0000","last_change":"2024-04-24T18:10:20.334440+0000","last_active":"2024-04-24T18:10:43.449727+0000","last_peered":"2024-04-24T18:10:43.449727+0000","last_clean":"2024-04-24T18:10:43.449727+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:10:43.449727+0000","last_undegraded":"2024-04-24T18:10:43.449727+0000","last_fullsized":"2024-04-24T18:10:43.449727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"167'1362","reported_seq":6969,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449833+0000","last_change":"2024-04-24T18:10:20.329528+0000","last_active":"2024-04-24T18:10:43.449833+0000","last_peered":"2024-04-24T18:10:43.449833+0000","last_clean":"2024-04-24T18:10:43.449833+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:10:43.449833+0000","last_undegraded":"2024-04-24T18:10:43.449833+0000","last_fullsized":"2024-04-24T18:10:43.449833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"174'1237","reported_seq":4971,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444127+0000","last_change":"2024-04-24T18:10:20.324905+0000","last_active":"2024-04-24T18:10:43.444127+0000","last_peered":"2024-04-24T18:10:43.444127+0000","last_clean":"2024-04-24T18:10:43.444127+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:10:43.444127+0000","last_undegraded":"2024-04-24T18:10:43.444127+0000","last_fullsized":"2024-04-24T18:10:43.444127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449538+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:10:43.449538+0000","last_peered":"2024-04-24T18:10:43.449538+0000","last_clean":"2024-04-24T18:10:43.449538+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:10:43.449538+0000","last_undegraded":"2024-04-24T18:10:43.449538+0000","last_fullsized":"2024-04-24T18:10:43.449538+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":177,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:18.331936+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:31.195614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"174'957","reported_seq":7180,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444736+0000","last_change":"2024-04-24T18:10:21.336769+0000","last_active":"2024-04-24T18:10:43.444736+0000","last_peered":"2024-04-24T18:10:43.444736+0000","last_clean":"2024-04-24T18:10:43.444736+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:43.444736+0000","last_undegraded":"2024-04-24T18:10:43.444736+0000","last_fullsized":"2024-04-24T18:10:43.444736+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:53:36.139560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449814+0000","last_change":"2024-04-24T18:10:18.330231+0000","last_active":"2024-04-24T18:10:43.449814+0000","last_peered":"2024-04-24T18:10:43.449814+0000","last_clean":"2024-04-24T18:10:43.449814+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:43.449814+0000","last_undegraded":"2024-04-24T18:10:43.449814+0000","last_fullsized":"2024-04-24T18:10:43.449814+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:00:41.033763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":204,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450179+0000","last_change":"2024-04-24T18:10:18.328460+0000","last_active":"2024-04-24T18:10:43.450179+0000","last_peered":"2024-04-24T18:10:43.450179+0000","last_clean":"2024-04-24T18:10:43.450179+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:43.450179+0000","last_undegraded":"2024-04-24T18:10:43.450179+0000","last_fullsized":"2024-04-24T18:10:43.450179+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:04.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"165'1061","reported_seq":6785,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444268+0000","last_change":"2024-04-24T18:10:20.326388+0000","last_active":"2024-04-24T18:10:43.444268+0000","last_peered":"2024-04-24T18:10:43.444268+0000","last_clean":"2024-04-24T18:10:43.444268+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:43.444268+0000","last_undegraded":"2024-04-24T18:10:43.444268+0000","last_fullsized":"2024-04-24T18:10:43.444268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"159'4","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449661+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:10:43.449661+0000","last_peered":"2024-04-24T18:10:43.449661+0000","last_clean":"2024-04-24T18:10:43.449661+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:43.449661+0000","last_undegraded":"2024-04-24T18:10:43.449661+0000","last_fullsized":"2024-04-24T18:10:43.449661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450353+0000","last_change":"2024-04-24T18:10:18.331833+0000","last_active":"2024-04-24T18:10:43.450353+0000","last_peered":"2024-04-24T18:10:43.450353+0000","last_clean":"2024-04-24T18:10:43.450353+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:43.450353+0000","last_undegraded":"2024-04-24T18:10:43.450353+0000","last_fullsized":"2024-04-24T18:10:43.450353+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:47.993822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"168'981","reported_seq":10279,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449459+0000","last_change":"2024-04-24T18:10:20.327837+0000","last_active":"2024-04-24T18:10:43.449459+0000","last_peered":"2024-04-24T18:10:43.449459+0000","last_clean":"2024-04-24T18:10:43.449459+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:10:43.449459+0000","last_undegraded":"2024-04-24T18:10:43.449459+0000","last_fullsized":"2024-04-24T18:10:43.449459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444134+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:10:43.444134+0000","last_peered":"2024-04-24T18:10:43.444134+0000","last_clean":"2024-04-24T18:10:43.444134+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:10:43.444134+0000","last_undegraded":"2024-04-24T18:10:43.444134+0000","last_fullsized":"2024-04-24T18:10:43.444134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450351+0000","last_change":"2024-04-24T18:10:18.331400+0000","last_active":"2024-04-24T18:10:43.450351+0000","last_peered":"2024-04-24T18:10:43.450351+0000","last_clean":"2024-04-24T18:10:43.450351+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:10:43.450351+0000","last_undegraded":"2024-04-24T18:10:43.450351+0000","last_fullsized":"2024-04-24T18:10:43.450351+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:18.213539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"167'574","reported_seq":5342,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444387+0000","last_change":"2024-04-24T18:10:20.327045+0000","last_active":"2024-04-24T18:10:43.444387+0000","last_peered":"2024-04-24T18:10:43.444387+0000","last_clean":"2024-04-24T18:10:43.444387+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:43.444387+0000","last_undegraded":"2024-04-24T18:10:43.444387+0000","last_fullsized":"2024-04-24T18:10:43.444387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"113'2","reported_seq":40,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444509+0000","last_change":"2024-04-24T18:10:18.333879+0000","last_active":"2024-04-24T18:10:43.444509+0000","last_peered":"2024-04-24T18:10:43.444509+0000","last_clean":"2024-04-24T18:10:43.444509+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:43.444509+0000","last_undegraded":"2024-04-24T18:10:43.444509+0000","last_fullsized":"2024-04-24T18:10:43.444509+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:18.541992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":381,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450453+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:10:43.450453+0000","last_peered":"2024-04-24T18:10:43.450453+0000","last_clean":"2024-04-24T18:10:43.450453+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:10:43.450453+0000","last_undegraded":"2024-04-24T18:10:43.450453+0000","last_fullsized":"2024-04-24T18:10:43.450453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444276+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:10:43.444276+0000","last_peered":"2024-04-24T18:10:43.444276+0000","last_clean":"2024-04-24T18:10:43.444276+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:10:43.444276+0000","last_undegraded":"2024-04-24T18:10:43.444276+0000","last_fullsized":"2024-04-24T18:10:43.444276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"174'1090","reported_seq":10185,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445190+0000","last_change":"2024-04-24T18:10:22.338092+0000","last_active":"2024-04-24T18:10:43.445190+0000","last_peered":"2024-04-24T18:10:43.445190+0000","last_clean":"2024-04-24T18:10:43.445190+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:10:43.445190+0000","last_undegraded":"2024-04-24T18:10:43.445190+0000","last_fullsized":"2024-04-24T18:10:43.445190+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:06:01.070097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"174'671","reported_seq":6166,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449838+0000","last_change":"2024-04-24T18:10:20.332296+0000","last_active":"2024-04-24T18:10:43.449838+0000","last_peered":"2024-04-24T18:10:43.449838+0000","last_clean":"2024-04-24T18:10:43.449838+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:43.449838+0000","last_undegraded":"2024-04-24T18:10:43.449838+0000","last_fullsized":"2024-04-24T18:10:43.449838+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":183,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449716+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:10:43.449716+0000","last_peered":"2024-04-24T18:10:43.449716+0000","last_clean":"2024-04-24T18:10:43.449716+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:43.449716+0000","last_undegraded":"2024-04-24T18:10:43.449716+0000","last_fullsized":"2024-04-24T18:10:43.449716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"171'818","reported_seq":6149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329213+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":186,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449934+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:10:43.449934+0000","last_peered":"2024-04-24T18:10:43.449934+0000","last_clean":"2024-04-24T18:10:43.449934+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:43.449934+0000","last_undegraded":"2024-04-24T18:10:43.449934+0000","last_fullsized":"2024-04-24T18:10:43.449934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"165'696","reported_seq":12152,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450131+0000","last_change":"2024-04-24T18:10:20.328757+0000","last_active":"2024-04-24T18:10:43.450131+0000","last_peered":"2024-04-24T18:10:43.450131+0000","last_clean":"2024-04-24T18:10:43.450131+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:10:43.450131+0000","last_undegraded":"2024-04-24T18:10:43.450131+0000","last_fullsized":"2024-04-24T18:10:43.450131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"171'666","reported_seq":6306,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449742+0000","last_change":"2024-04-24T18:10:20.331718+0000","last_active":"2024-04-24T18:10:43.449742+0000","last_peered":"2024-04-24T18:10:43.449742+0000","last_clean":"2024-04-24T18:10:43.449742+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:43.449742+0000","last_undegraded":"2024-04-24T18:10:43.449742+0000","last_fullsized":"2024-04-24T18:10:43.449742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"167'694","reported_seq":3935,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:20.328410+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"171'682","reported_seq":6313,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444827+0000","last_change":"2024-04-24T18:10:21.332171+0000","last_active":"2024-04-24T18:10:43.444827+0000","last_peered":"2024-04-24T18:10:43.444827+0000","last_clean":"2024-04-24T18:10:43.444827+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:43.444827+0000","last_undegraded":"2024-04-24T18:10:43.444827+0000","last_fullsized":"2024-04-24T18:10:43.444827+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:41.461799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"171'580","reported_seq":7072,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449978+0000","last_change":"2024-04-24T18:10:20.332029+0000","last_active":"2024-04-24T18:10:43.449978+0000","last_peered":"2024-04-24T18:10:43.449978+0000","last_clean":"2024-04-24T18:10:43.449978+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:10:43.449978+0000","last_undegraded":"2024-04-24T18:10:43.449978+0000","last_fullsized":"2024-04-24T18:10:43.449978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"167'634","reported_seq":9280,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449993+0000","last_change":"2024-04-24T18:10:20.334138+0000","last_active":"2024-04-24T18:10:43.449993+0000","last_peered":"2024-04-24T18:10:43.449993+0000","last_clean":"2024-04-24T18:10:43.449993+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:43.449993+0000","last_undegraded":"2024-04-24T18:10:43.449993+0000","last_fullsized":"2024-04-24T18:10:43.449993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"167'596","reported_seq":7179,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449905+0000","last_change":"2024-04-24T18:10:20.334800+0000","last_active":"2024-04-24T18:10:43.449905+0000","last_peered":"2024-04-24T18:10:43.449905+0000","last_clean":"2024-04-24T18:10:43.449905+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:43.449905+0000","last_undegraded":"2024-04-24T18:10:43.449905+0000","last_fullsized":"2024-04-24T18:10:43.449905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"167'868","reported_seq":6431,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445558+0000","last_change":"2024-04-24T18:10:20.326786+0000","last_active":"2024-04-24T18:10:43.445558+0000","last_peered":"2024-04-24T18:10:43.445558+0000","last_clean":"2024-04-24T18:10:43.445558+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:10:43.445558+0000","last_undegraded":"2024-04-24T18:10:43.445558+0000","last_fullsized":"2024-04-24T18:10:43.445558+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"167'663","reported_seq":7482,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449450+0000","last_change":"2024-04-24T18:10:20.329149+0000","last_active":"2024-04-24T18:10:43.449450+0000","last_peered":"2024-04-24T18:10:43.449450+0000","last_clean":"2024-04-24T18:10:43.449450+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:43.449450+0000","last_undegraded":"2024-04-24T18:10:43.449450+0000","last_fullsized":"2024-04-24T18:10:43.449450+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"171'765","reported_seq":9197,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329631+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"174'662","reported_seq":6522,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444247+0000","last_change":"2024-04-24T18:10:20.325240+0000","last_active":"2024-04-24T18:10:43.444247+0000","last_peered":"2024-04-24T18:10:43.444247+0000","last_clean":"2024-04-24T18:10:43.444247+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:43.444247+0000","last_undegraded":"2024-04-24T18:10:43.444247+0000","last_fullsized":"2024-04-24T18:10:43.444247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"167'967","reported_seq":9149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449919+0000","last_change":"2024-04-24T18:10:20.334115+0000","last_active":"2024-04-24T18:10:43.449919+0000","last_peered":"2024-04-24T18:10:43.449919+0000","last_clean":"2024-04-24T18:10:43.449919+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:10:43.449919+0000","last_undegraded":"2024-04-24T18:10:43.449919+0000","last_fullsized":"2024-04-24T18:10:43.449919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"167'688","reported_seq":8040,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449785+0000","last_change":"2024-04-24T18:10:20.330052+0000","last_active":"2024-04-24T18:10:43.449785+0000","last_peered":"2024-04-24T18:10:43.449785+0000","last_clean":"2024-04-24T18:10:43.449785+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:10:43.449785+0000","last_undegraded":"2024-04-24T18:10:43.449785+0000","last_fullsized":"2024-04-24T18:10:43.449785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"167'621","reported_seq":6044,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450254+0000","last_change":"2024-04-24T18:10:20.329098+0000","last_active":"2024-04-24T18:10:43.450254+0000","last_peered":"2024-04-24T18:10:43.450254+0000","last_clean":"2024-04-24T18:10:43.450254+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:10:43.450254+0000","last_undegraded":"2024-04-24T18:10:43.450254+0000","last_fullsized":"2024-04-24T18:10:43.450254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"171'1163","reported_seq":7009,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445089+0000","last_change":"2024-04-24T18:10:22.337859+0000","last_active":"2024-04-24T18:10:43.445089+0000","last_peered":"2024-04-24T18:10:43.445089+0000","last_clean":"2024-04-24T18:10:43.445089+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:43.445089+0000","last_undegraded":"2024-04-24T18:10:43.445089+0000","last_fullsized":"2024-04-24T18:10:43.445089+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:55.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":26264,"ondisk_log_size":26264,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307912,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063928,"statfs":{"total":96636764160,"available":96321462272,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53000000000000003,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48599999999999999,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.67000000000000004,"5min":2.585,"15min":2.585},"last":0.64800000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78300000000000003,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":2.008,"15min":2.008},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.55100000000000005,"15min":0.51000000000000001},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.42999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.753,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":1.196,"15min":1.196},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":1.131,"15min":1.131},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77900000000000003,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75700000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893145,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309128,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":306784,"kb_avail":94062712,"statfs":{"total":96636764160,"available":96320217088,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":314146847},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":10.071999999999999,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.38}]},{"osd":2,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":2.9609999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.498},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307888,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063952,"statfs":{"total":96636764160,"available":96321486848,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.53500000000000003,"15min":0.61799999999999999},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.5,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.89500000000000002,"15min":1.127},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":14.15,"15min":14.15},"last":0.28199999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.93600000000000005,"5min":1.306,"15min":1.306},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.96099999999999997,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.95499999999999996,"5min":14.118,"15min":14.118},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.93000000000000005,"15min":1.1519999999999999},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67600000000000005,"5min":14.089,"15min":14.089},"last":0.371},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.313}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":0.61099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71799999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.0600000000000001,"15min":1.274},"min":{"1min":0.35699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.76300000000000001},"min":{"1min":0.371,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2310000000000001,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.746,"5min":1.49,"15min":1.49},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.073,"15min":1.2929999999999999},"min":{"1min":0.438,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1279999999999999,"5min":14.051,"15min":14.051},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.74399999999999999,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.1870000000000001,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311956,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059884,"statfs":{"total":96636764160,"available":96317321216,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65700000000000003,"5min":1.375,"15min":1.375},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":1.2549999999999999,"15min":1.6379999999999999},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90200000000000002,"5min":26.936,"15min":26.936},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.92000000000000004,"15min":1.1479999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":11.93,"15min":11.93},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42699999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.157,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.83599999999999997,"15min":0.995},"min":{"1min":0.25900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.93799999999999994,"5min":14.371,"15min":14.371},"last":0.432},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.86699999999999999,"15min":0.97399999999999998},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.92800000000000005,"15min":1.1140000000000001},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":14.414,"15min":14.414},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.93200000000000005,"5min":1.329,"15min":1.329},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83099999999999996,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97099999999999997,"15min":1.133},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.995,"5min":14.452,"15min":14.452},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86099999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.98699999999999999,"15min":1.1479999999999999},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1259999999999999,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299652,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072188,"statfs":{"total":96636764160,"available":96329920512,"internally_reserved":0,"allocated":2342912,"data_stored":1194912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":4.109,"15min":4.109},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.441,"15min":0.42099999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77900000000000003,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.434,"15min":0.40699999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":1.619,"15min":1.619},"last":0.23699999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.316}]},{"osd":4,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":1.155,"15min":1.155},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.71799999999999997,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":10.516,"15min":10.516},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.56999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2355200,"data_stored":1195101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36299999999999999,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.41799999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":2.2370000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.86699999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48099999999999998,"5min":1.792,"15min":1.792},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":4.1779999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":4.0899999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.44800000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.50900000000000001,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":3.044}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68400000000000005,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.76300000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":2.323},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70699999999999996,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.96499999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":2.282},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.755,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":2.3559999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.74299999999999999,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":15.4,"15min":15.4},"last":2.3849999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311952,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059888,"statfs":{"total":96636764160,"available":96317325312,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46800000000000003,"15min":0.47499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.497},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70699999999999996}]},{"osd":1,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55000000000000004,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.626,"5min":0.627,"15min":0.627},"last":0.72299999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.33900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.27600000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63800000000000001,"5min":1.153,"15min":1.153},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":1.373,"15min":1.373},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51800000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71099999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303756,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068084,"statfs":{"total":96636764160,"available":96325718016,"internally_reserved":0,"allocated":2351104,"data_stored":1195082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.307,"5min":0.189,"15min":0.189},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":1.296,"15min":1.296},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.44,"5min":0.504,"15min":0.53100000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63300000000000001,"5min":1.377,"15min":1.377},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":1.899,"15min":1.899},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59899999999999998,"5min":1.881,"15min":1.881},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46999999999999997,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.66100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52000000000000002,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.754}]},{"osd":7,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":1.016,"15min":1.016},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54400000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.70299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:10:48.308 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:10:48.642 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:48.643 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:10:48.655 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-04-24T18:10:47.337398+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590640,"num_objects":1800,"num_object_clones":0,"num_object_copies":3600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29264,"num_write_kb":534157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26387,"ondisk_log_size":26387,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456004,"kb_used_data":18492,"kb_used_omap":43,"kb_used_meta":2437332,"kb_avail":752518716,"statfs":{"total":773094113280,"available":770579165184,"internally_reserved":0,"allocated":18935808,"data_stored":10771526,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2495828359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.905362"},"pg_stats":[{"pgid":"4.1f","version":"173'683","reported_seq":7230,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449490+0000","last_change":"2024-04-24T18:10:27.328087+0000","last_active":"2024-04-24T18:10:43.449490+0000","last_peered":"2024-04-24T18:10:43.449490+0000","last_clean":"2024-04-24T18:10:43.449490+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:10:43.449490+0000","last_undegraded":"2024-04-24T18:10:43.449490+0000","last_fullsized":"2024-04-24T18:10:43.449490+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:17.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"160'1339","reported_seq":7257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445000+0000","last_change":"2024-04-24T18:10:21.335419+0000","last_active":"2024-04-24T18:10:43.445000+0000","last_peered":"2024-04-24T18:10:43.445000+0000","last_clean":"2024-04-24T18:10:43.445000+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:43.445000+0000","last_undegraded":"2024-04-24T18:10:43.445000+0000","last_fullsized":"2024-04-24T18:10:43.445000+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1339,"log_dups_size":0,"ondisk_log_size":1339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:03.319026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"167'710","reported_seq":5997,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449714+0000","last_change":"2024-04-24T18:10:20.330289+0000","last_active":"2024-04-24T18:10:43.449714+0000","last_peered":"2024-04-24T18:10:43.449714+0000","last_clean":"2024-04-24T18:10:43.449714+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:10:43.449714+0000","last_undegraded":"2024-04-24T18:10:43.449714+0000","last_fullsized":"2024-04-24T18:10:43.449714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1079","reported_seq":10264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445678+0000","last_change":"2024-04-24T18:10:20.345289+0000","last_active":"2024-04-24T18:10:43.445678+0000","last_peered":"2024-04-24T18:10:43.445678+0000","last_clean":"2024-04-24T18:10:43.445678+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:43.445678+0000","last_undegraded":"2024-04-24T18:10:43.445678+0000","last_fullsized":"2024-04-24T18:10:43.445678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"171'535","reported_seq":11656,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449807+0000","last_change":"2024-04-24T18:10:20.332101+0000","last_active":"2024-04-24T18:10:43.449807+0000","last_peered":"2024-04-24T18:10:43.449807+0000","last_clean":"2024-04-24T18:10:43.449807+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:43.449807+0000","last_undegraded":"2024-04-24T18:10:43.449807+0000","last_fullsized":"2024-04-24T18:10:43.449807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"167'880","reported_seq":12337,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449756+0000","last_change":"2024-04-24T18:10:20.331679+0000","last_active":"2024-04-24T18:10:43.449756+0000","last_peered":"2024-04-24T18:10:43.449756+0000","last_clean":"2024-04-24T18:10:43.449756+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:43.449756+0000","last_undegraded":"2024-04-24T18:10:43.449756+0000","last_fullsized":"2024-04-24T18:10:43.449756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"167'726","reported_seq":9735,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449505+0000","last_change":"2024-04-24T18:10:20.329954+0000","last_active":"2024-04-24T18:10:43.449505+0000","last_peered":"2024-04-24T18:10:43.449505+0000","last_clean":"2024-04-24T18:10:43.449505+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:43.449505+0000","last_undegraded":"2024-04-24T18:10:43.449505+0000","last_fullsized":"2024-04-24T18:10:43.449505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"167'616","reported_seq":11663,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449727+0000","last_change":"2024-04-24T18:10:20.334440+0000","last_active":"2024-04-24T18:10:43.449727+0000","last_peered":"2024-04-24T18:10:43.449727+0000","last_clean":"2024-04-24T18:10:43.449727+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:10:43.449727+0000","last_undegraded":"2024-04-24T18:10:43.449727+0000","last_fullsized":"2024-04-24T18:10:43.449727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"167'1362","reported_seq":6969,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449833+0000","last_change":"2024-04-24T18:10:20.329528+0000","last_active":"2024-04-24T18:10:43.449833+0000","last_peered":"2024-04-24T18:10:43.449833+0000","last_clean":"2024-04-24T18:10:43.449833+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:10:43.449833+0000","last_undegraded":"2024-04-24T18:10:43.449833+0000","last_fullsized":"2024-04-24T18:10:43.449833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"174'1237","reported_seq":4971,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444127+0000","last_change":"2024-04-24T18:10:20.324905+0000","last_active":"2024-04-24T18:10:43.444127+0000","last_peered":"2024-04-24T18:10:43.444127+0000","last_clean":"2024-04-24T18:10:43.444127+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:10:43.444127+0000","last_undegraded":"2024-04-24T18:10:43.444127+0000","last_fullsized":"2024-04-24T18:10:43.444127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449538+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:10:43.449538+0000","last_peered":"2024-04-24T18:10:43.449538+0000","last_clean":"2024-04-24T18:10:43.449538+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:10:43.449538+0000","last_undegraded":"2024-04-24T18:10:43.449538+0000","last_fullsized":"2024-04-24T18:10:43.449538+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":177,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:18.331936+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:31.195614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"174'957","reported_seq":7180,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444736+0000","last_change":"2024-04-24T18:10:21.336769+0000","last_active":"2024-04-24T18:10:43.444736+0000","last_peered":"2024-04-24T18:10:43.444736+0000","last_clean":"2024-04-24T18:10:43.444736+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:43.444736+0000","last_undegraded":"2024-04-24T18:10:43.444736+0000","last_fullsized":"2024-04-24T18:10:43.444736+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:53:36.139560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449814+0000","last_change":"2024-04-24T18:10:18.330231+0000","last_active":"2024-04-24T18:10:43.449814+0000","last_peered":"2024-04-24T18:10:43.449814+0000","last_clean":"2024-04-24T18:10:43.449814+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:43.449814+0000","last_undegraded":"2024-04-24T18:10:43.449814+0000","last_fullsized":"2024-04-24T18:10:43.449814+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:00:41.033763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":204,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450179+0000","last_change":"2024-04-24T18:10:18.328460+0000","last_active":"2024-04-24T18:10:43.450179+0000","last_peered":"2024-04-24T18:10:43.450179+0000","last_clean":"2024-04-24T18:10:43.450179+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:43.450179+0000","last_undegraded":"2024-04-24T18:10:43.450179+0000","last_fullsized":"2024-04-24T18:10:43.450179+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:04.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"165'1061","reported_seq":6785,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444268+0000","last_change":"2024-04-24T18:10:20.326388+0000","last_active":"2024-04-24T18:10:43.444268+0000","last_peered":"2024-04-24T18:10:43.444268+0000","last_clean":"2024-04-24T18:10:43.444268+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:43.444268+0000","last_undegraded":"2024-04-24T18:10:43.444268+0000","last_fullsized":"2024-04-24T18:10:43.444268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"159'4","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449661+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:10:43.449661+0000","last_peered":"2024-04-24T18:10:43.449661+0000","last_clean":"2024-04-24T18:10:43.449661+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:43.449661+0000","last_undegraded":"2024-04-24T18:10:43.449661+0000","last_fullsized":"2024-04-24T18:10:43.449661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450353+0000","last_change":"2024-04-24T18:10:18.331833+0000","last_active":"2024-04-24T18:10:43.450353+0000","last_peered":"2024-04-24T18:10:43.450353+0000","last_clean":"2024-04-24T18:10:43.450353+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:43.450353+0000","last_undegraded":"2024-04-24T18:10:43.450353+0000","last_fullsized":"2024-04-24T18:10:43.450353+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:47.993822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"168'981","reported_seq":10279,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449459+0000","last_change":"2024-04-24T18:10:20.327837+0000","last_active":"2024-04-24T18:10:43.449459+0000","last_peered":"2024-04-24T18:10:43.449459+0000","last_clean":"2024-04-24T18:10:43.449459+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:10:43.449459+0000","last_undegraded":"2024-04-24T18:10:43.449459+0000","last_fullsized":"2024-04-24T18:10:43.449459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444134+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:10:43.444134+0000","last_peered":"2024-04-24T18:10:43.444134+0000","last_clean":"2024-04-24T18:10:43.444134+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:10:43.444134+0000","last_undegraded":"2024-04-24T18:10:43.444134+0000","last_fullsized":"2024-04-24T18:10:43.444134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450351+0000","last_change":"2024-04-24T18:10:18.331400+0000","last_active":"2024-04-24T18:10:43.450351+0000","last_peered":"2024-04-24T18:10:43.450351+0000","last_clean":"2024-04-24T18:10:43.450351+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:10:43.450351+0000","last_undegraded":"2024-04-24T18:10:43.450351+0000","last_fullsized":"2024-04-24T18:10:43.450351+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:18.213539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"167'574","reported_seq":5342,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444387+0000","last_change":"2024-04-24T18:10:20.327045+0000","last_active":"2024-04-24T18:10:43.444387+0000","last_peered":"2024-04-24T18:10:43.444387+0000","last_clean":"2024-04-24T18:10:43.444387+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:43.444387+0000","last_undegraded":"2024-04-24T18:10:43.444387+0000","last_fullsized":"2024-04-24T18:10:43.444387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"113'2","reported_seq":40,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444509+0000","last_change":"2024-04-24T18:10:18.333879+0000","last_active":"2024-04-24T18:10:43.444509+0000","last_peered":"2024-04-24T18:10:43.444509+0000","last_clean":"2024-04-24T18:10:43.444509+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:43.444509+0000","last_undegraded":"2024-04-24T18:10:43.444509+0000","last_fullsized":"2024-04-24T18:10:43.444509+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:18.541992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":381,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450453+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:10:43.450453+0000","last_peered":"2024-04-24T18:10:43.450453+0000","last_clean":"2024-04-24T18:10:43.450453+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:10:43.450453+0000","last_undegraded":"2024-04-24T18:10:43.450453+0000","last_fullsized":"2024-04-24T18:10:43.450453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444276+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:10:43.444276+0000","last_peered":"2024-04-24T18:10:43.444276+0000","last_clean":"2024-04-24T18:10:43.444276+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:10:43.444276+0000","last_undegraded":"2024-04-24T18:10:43.444276+0000","last_fullsized":"2024-04-24T18:10:43.444276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"174'1090","reported_seq":10185,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445190+0000","last_change":"2024-04-24T18:10:22.338092+0000","last_active":"2024-04-24T18:10:43.445190+0000","last_peered":"2024-04-24T18:10:43.445190+0000","last_clean":"2024-04-24T18:10:43.445190+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:10:43.445190+0000","last_undegraded":"2024-04-24T18:10:43.445190+0000","last_fullsized":"2024-04-24T18:10:43.445190+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:06:01.070097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"174'671","reported_seq":6166,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449838+0000","last_change":"2024-04-24T18:10:20.332296+0000","last_active":"2024-04-24T18:10:43.449838+0000","last_peered":"2024-04-24T18:10:43.449838+0000","last_clean":"2024-04-24T18:10:43.449838+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:43.449838+0000","last_undegraded":"2024-04-24T18:10:43.449838+0000","last_fullsized":"2024-04-24T18:10:43.449838+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":183,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449716+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:10:43.449716+0000","last_peered":"2024-04-24T18:10:43.449716+0000","last_clean":"2024-04-24T18:10:43.449716+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:43.449716+0000","last_undegraded":"2024-04-24T18:10:43.449716+0000","last_fullsized":"2024-04-24T18:10:43.449716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"171'818","reported_seq":6149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329213+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":186,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449934+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:10:43.449934+0000","last_peered":"2024-04-24T18:10:43.449934+0000","last_clean":"2024-04-24T18:10:43.449934+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:43.449934+0000","last_undegraded":"2024-04-24T18:10:43.449934+0000","last_fullsized":"2024-04-24T18:10:43.449934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"165'696","reported_seq":12152,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450131+0000","last_change":"2024-04-24T18:10:20.328757+0000","last_active":"2024-04-24T18:10:43.450131+0000","last_peered":"2024-04-24T18:10:43.450131+0000","last_clean":"2024-04-24T18:10:43.450131+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:10:43.450131+0000","last_undegraded":"2024-04-24T18:10:43.450131+0000","last_fullsized":"2024-04-24T18:10:43.450131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"171'666","reported_seq":6306,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449742+0000","last_change":"2024-04-24T18:10:20.331718+0000","last_active":"2024-04-24T18:10:43.449742+0000","last_peered":"2024-04-24T18:10:43.449742+0000","last_clean":"2024-04-24T18:10:43.449742+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:43.449742+0000","last_undegraded":"2024-04-24T18:10:43.449742+0000","last_fullsized":"2024-04-24T18:10:43.449742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"167'694","reported_seq":3935,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:20.328410+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"171'682","reported_seq":6313,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444827+0000","last_change":"2024-04-24T18:10:21.332171+0000","last_active":"2024-04-24T18:10:43.444827+0000","last_peered":"2024-04-24T18:10:43.444827+0000","last_clean":"2024-04-24T18:10:43.444827+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:43.444827+0000","last_undegraded":"2024-04-24T18:10:43.444827+0000","last_fullsized":"2024-04-24T18:10:43.444827+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:41.461799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"171'580","reported_seq":7072,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449978+0000","last_change":"2024-04-24T18:10:20.332029+0000","last_active":"2024-04-24T18:10:43.449978+0000","last_peered":"2024-04-24T18:10:43.449978+0000","last_clean":"2024-04-24T18:10:43.449978+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:10:43.449978+0000","last_undegraded":"2024-04-24T18:10:43.449978+0000","last_fullsized":"2024-04-24T18:10:43.449978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"167'634","reported_seq":9280,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449993+0000","last_change":"2024-04-24T18:10:20.334138+0000","last_active":"2024-04-24T18:10:43.449993+0000","last_peered":"2024-04-24T18:10:43.449993+0000","last_clean":"2024-04-24T18:10:43.449993+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:43.449993+0000","last_undegraded":"2024-04-24T18:10:43.449993+0000","last_fullsized":"2024-04-24T18:10:43.449993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"167'596","reported_seq":7179,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449905+0000","last_change":"2024-04-24T18:10:20.334800+0000","last_active":"2024-04-24T18:10:43.449905+0000","last_peered":"2024-04-24T18:10:43.449905+0000","last_clean":"2024-04-24T18:10:43.449905+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:43.449905+0000","last_undegraded":"2024-04-24T18:10:43.449905+0000","last_fullsized":"2024-04-24T18:10:43.449905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"167'868","reported_seq":6431,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445558+0000","last_change":"2024-04-24T18:10:20.326786+0000","last_active":"2024-04-24T18:10:43.445558+0000","last_peered":"2024-04-24T18:10:43.445558+0000","last_clean":"2024-04-24T18:10:43.445558+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:10:43.445558+0000","last_undegraded":"2024-04-24T18:10:43.445558+0000","last_fullsized":"2024-04-24T18:10:43.445558+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"167'663","reported_seq":7482,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449450+0000","last_change":"2024-04-24T18:10:20.329149+0000","last_active":"2024-04-24T18:10:43.449450+0000","last_peered":"2024-04-24T18:10:43.449450+0000","last_clean":"2024-04-24T18:10:43.449450+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:43.449450+0000","last_undegraded":"2024-04-24T18:10:43.449450+0000","last_fullsized":"2024-04-24T18:10:43.449450+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"171'765","reported_seq":9197,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329631+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"174'662","reported_seq":6522,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444247+0000","last_change":"2024-04-24T18:10:20.325240+0000","last_active":"2024-04-24T18:10:43.444247+0000","last_peered":"2024-04-24T18:10:43.444247+0000","last_clean":"2024-04-24T18:10:43.444247+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:43.444247+0000","last_undegraded":"2024-04-24T18:10:43.444247+0000","last_fullsized":"2024-04-24T18:10:43.444247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"167'967","reported_seq":9149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449919+0000","last_change":"2024-04-24T18:10:20.334115+0000","last_active":"2024-04-24T18:10:43.449919+0000","last_peered":"2024-04-24T18:10:43.449919+0000","last_clean":"2024-04-24T18:10:43.449919+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:10:43.449919+0000","last_undegraded":"2024-04-24T18:10:43.449919+0000","last_fullsized":"2024-04-24T18:10:43.449919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"167'688","reported_seq":8040,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449785+0000","last_change":"2024-04-24T18:10:20.330052+0000","last_active":"2024-04-24T18:10:43.449785+0000","last_peered":"2024-04-24T18:10:43.449785+0000","last_clean":"2024-04-24T18:10:43.449785+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:10:43.449785+0000","last_undegraded":"2024-04-24T18:10:43.449785+0000","last_fullsized":"2024-04-24T18:10:43.449785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"167'621","reported_seq":6044,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450254+0000","last_change":"2024-04-24T18:10:20.329098+0000","last_active":"2024-04-24T18:10:43.450254+0000","last_peered":"2024-04-24T18:10:43.450254+0000","last_clean":"2024-04-24T18:10:43.450254+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:10:43.450254+0000","last_undegraded":"2024-04-24T18:10:43.450254+0000","last_fullsized":"2024-04-24T18:10:43.450254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"171'1163","reported_seq":7009,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445089+0000","last_change":"2024-04-24T18:10:22.337859+0000","last_active":"2024-04-24T18:10:43.445089+0000","last_peered":"2024-04-24T18:10:43.445089+0000","last_clean":"2024-04-24T18:10:43.445089+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:43.445089+0000","last_undegraded":"2024-04-24T18:10:43.445089+0000","last_fullsized":"2024-04-24T18:10:43.445089+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:55.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":26264,"ondisk_log_size":26264,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574893,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307912,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063928,"statfs":{"total":96636764160,"available":96321462272,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53000000000000003,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48599999999999999,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.67000000000000004,"5min":2.585,"15min":2.585},"last":0.64800000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78300000000000003,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":2.008,"15min":2.008},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.55100000000000005,"15min":0.51000000000000001},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.42999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.753,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":1.196,"15min":1.196},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":1.131,"15min":1.131},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77900000000000003,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75700000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893145,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309128,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":306784,"kb_avail":94062712,"statfs":{"total":96636764160,"available":96320217088,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":314146847},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":10.071999999999999,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.38}]},{"osd":2,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":2.9609999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.498},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574892,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307888,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063952,"statfs":{"total":96636764160,"available":96321486848,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.53500000000000003,"15min":0.61799999999999999},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.5,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.89500000000000002,"15min":1.127},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":14.15,"15min":14.15},"last":0.28199999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.93600000000000005,"5min":1.306,"15min":1.306},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.96099999999999997,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.95499999999999996,"5min":14.118,"15min":14.118},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.93000000000000005,"15min":1.1519999999999999},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67600000000000005,"5min":14.089,"15min":14.089},"last":0.371},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.313}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":0.61099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71799999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.0600000000000001,"15min":1.274},"min":{"1min":0.35699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.76300000000000001},"min":{"1min":0.371,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2310000000000001,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.746,"5min":1.49,"15min":1.49},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.073,"15min":1.2929999999999999},"min":{"1min":0.438,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1279999999999999,"5min":14.051,"15min":14.051},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.74399999999999999,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.1870000000000001,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574893,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311956,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059884,"statfs":{"total":96636764160,"available":96317321216,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65700000000000003,"5min":1.375,"15min":1.375},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":1.2549999999999999,"15min":1.6379999999999999},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90200000000000002,"5min":26.936,"15min":26.936},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.92000000000000004,"15min":1.1479999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":11.93,"15min":11.93},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42699999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.157,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.83599999999999997,"15min":0.995},"min":{"1min":0.25900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.93799999999999994,"5min":14.371,"15min":14.371},"last":0.432},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.86699999999999999,"15min":0.97399999999999998},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.92800000000000005,"15min":1.1140000000000001},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":14.414,"15min":14.414},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.93200000000000005,"5min":1.329,"15min":1.329},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83099999999999996,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97099999999999997,"15min":1.133},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.995,"5min":14.452,"15min":14.452},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86099999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.98699999999999999,"15min":1.1479999999999999},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1259999999999999,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574892,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299652,"kb_used_data":2288,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072188,"statfs":{"total":96636764160,"available":96329920512,"internally_reserved":0,"allocated":2342912,"data_stored":1194912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":4.109,"15min":4.109},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.441,"15min":0.42099999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77900000000000003,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.434,"15min":0.40699999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":1.619,"15min":1.619},"last":0.23699999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.316}]},{"osd":4,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":1.155,"15min":1.155},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.71799999999999997,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":10.516,"15min":10.516},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.56999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574893,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2355200,"data_stored":1195101,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36299999999999999,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.41799999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":2.2370000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.86699999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48099999999999998,"5min":1.792,"15min":1.792},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":4.1779999999999999}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":4.0899999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.44800000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.50900000000000001,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":3.044}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68400000000000005,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.76300000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":2.323},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70699999999999996,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.96499999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":2.282},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.93700000000000006}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.755,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":2.3559999999999999},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.74299999999999999,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":15.4,"15min":15.4},"last":2.3849999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574893,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311952,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059888,"statfs":{"total":96636764160,"available":96317325312,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46800000000000003,"15min":0.47499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.497},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.70699999999999996}]},{"osd":1,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55000000000000004,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.626,"5min":0.627,"15min":0.627},"last":0.72299999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.33900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.27600000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63800000000000001,"5min":1.153,"15min":1.153},"last":0.66200000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.59099999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":1.373,"15min":1.373},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.51800000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.57399999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71099999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.67900000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574893,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303756,"kb_used_data":2296,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068084,"statfs":{"total":96636764160,"available":96325718016,"internally_reserved":0,"allocated":2351104,"data_stored":1195082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.307,"5min":0.189,"15min":0.189},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.41899999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.68799999999999994}]},{"osd":2,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":1.296,"15min":1.296},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.44,"5min":0.504,"15min":0.53100000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63300000000000001,"5min":1.377,"15min":1.377},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":1.899,"15min":1.899},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59899999999999998,"5min":1.881,"15min":1.881},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46999999999999997,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.66100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52000000000000002,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.754}]},{"osd":7,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":1.016,"15min":1.016},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54400000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.70299999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:10:48.658 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-04-24T18:10:48.658 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-04-24T18:10:49.007 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-24T18:10:49.016 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2024-04-24T18:10:49.016 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:10:49.346 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:49.346 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":195,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:10:44.445859+0000","last_up_change":"2024-04-24T18:08:52.865354+0000","last_in_change":"2024-04-24T18:10:17.313880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-24T18:07:18.850062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-24T18:07:19.353026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T18:07:25.800822+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":1,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T18:07:35.513947+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":107,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":95,"up_thru":178,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":58122},{"type":"v1","addr":"172.21.15.78:6825","nonce":58122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":58122},{"type":"v1","addr":"172.21.15.78:6827","nonce":58122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":58122},{"type":"v1","addr":"172.21.15.78:6831","nonce":58122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":58122},{"type":"v1","addr":"172.21.15.78:6829","nonce":58122}]},"public_addr":"172.21.15.78:6825/58122","cluster_addr":"172.21.15.78:6827/58122","heartbeat_back_addr":"172.21.15.78:6831/58122","heartbeat_front_addr":"172.21.15.78:6829/58122","state":["exists","up"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":69},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:10:49.360 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-24T18:10:49.361 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:10:49.696 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:49.697 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:10:49.709 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-24T18:10:49.337839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590640,"num_objects":1800,"num_object_clones":0,"num_object_copies":3600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29264,"num_write_kb":534157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26387,"ondisk_log_size":26387,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456016,"kb_used_data":18528,"kb_used_omap":43,"kb_used_meta":2437332,"kb_avail":752518704,"statfs":{"total":773094113280,"available":770579152896,"internally_reserved":0,"allocated":18972672,"data_stored":10790096,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2495828359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.900840"},"pg_stats":[{"pgid":"4.1f","version":"173'683","reported_seq":7230,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449490+0000","last_change":"2024-04-24T18:10:27.328087+0000","last_active":"2024-04-24T18:10:43.449490+0000","last_peered":"2024-04-24T18:10:43.449490+0000","last_clean":"2024-04-24T18:10:43.449490+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:10:43.449490+0000","last_undegraded":"2024-04-24T18:10:43.449490+0000","last_fullsized":"2024-04-24T18:10:43.449490+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:17.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"160'1339","reported_seq":7257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445000+0000","last_change":"2024-04-24T18:10:21.335419+0000","last_active":"2024-04-24T18:10:43.445000+0000","last_peered":"2024-04-24T18:10:43.445000+0000","last_clean":"2024-04-24T18:10:43.445000+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:43.445000+0000","last_undegraded":"2024-04-24T18:10:43.445000+0000","last_fullsized":"2024-04-24T18:10:43.445000+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1339,"log_dups_size":0,"ondisk_log_size":1339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:03.319026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"167'710","reported_seq":5997,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449714+0000","last_change":"2024-04-24T18:10:20.330289+0000","last_active":"2024-04-24T18:10:43.449714+0000","last_peered":"2024-04-24T18:10:43.449714+0000","last_clean":"2024-04-24T18:10:43.449714+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:10:43.449714+0000","last_undegraded":"2024-04-24T18:10:43.449714+0000","last_fullsized":"2024-04-24T18:10:43.449714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1079","reported_seq":10264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445678+0000","last_change":"2024-04-24T18:10:20.345289+0000","last_active":"2024-04-24T18:10:43.445678+0000","last_peered":"2024-04-24T18:10:43.445678+0000","last_clean":"2024-04-24T18:10:43.445678+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:43.445678+0000","last_undegraded":"2024-04-24T18:10:43.445678+0000","last_fullsized":"2024-04-24T18:10:43.445678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"171'535","reported_seq":11656,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449807+0000","last_change":"2024-04-24T18:10:20.332101+0000","last_active":"2024-04-24T18:10:43.449807+0000","last_peered":"2024-04-24T18:10:43.449807+0000","last_clean":"2024-04-24T18:10:43.449807+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:43.449807+0000","last_undegraded":"2024-04-24T18:10:43.449807+0000","last_fullsized":"2024-04-24T18:10:43.449807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"167'880","reported_seq":12337,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449756+0000","last_change":"2024-04-24T18:10:20.331679+0000","last_active":"2024-04-24T18:10:43.449756+0000","last_peered":"2024-04-24T18:10:43.449756+0000","last_clean":"2024-04-24T18:10:43.449756+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:43.449756+0000","last_undegraded":"2024-04-24T18:10:43.449756+0000","last_fullsized":"2024-04-24T18:10:43.449756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"167'726","reported_seq":9735,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449505+0000","last_change":"2024-04-24T18:10:20.329954+0000","last_active":"2024-04-24T18:10:43.449505+0000","last_peered":"2024-04-24T18:10:43.449505+0000","last_clean":"2024-04-24T18:10:43.449505+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:43.449505+0000","last_undegraded":"2024-04-24T18:10:43.449505+0000","last_fullsized":"2024-04-24T18:10:43.449505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"167'616","reported_seq":11663,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449727+0000","last_change":"2024-04-24T18:10:20.334440+0000","last_active":"2024-04-24T18:10:43.449727+0000","last_peered":"2024-04-24T18:10:43.449727+0000","last_clean":"2024-04-24T18:10:43.449727+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:10:43.449727+0000","last_undegraded":"2024-04-24T18:10:43.449727+0000","last_fullsized":"2024-04-24T18:10:43.449727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"167'1362","reported_seq":6969,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449833+0000","last_change":"2024-04-24T18:10:20.329528+0000","last_active":"2024-04-24T18:10:43.449833+0000","last_peered":"2024-04-24T18:10:43.449833+0000","last_clean":"2024-04-24T18:10:43.449833+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:10:43.449833+0000","last_undegraded":"2024-04-24T18:10:43.449833+0000","last_fullsized":"2024-04-24T18:10:43.449833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"174'1237","reported_seq":4971,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444127+0000","last_change":"2024-04-24T18:10:20.324905+0000","last_active":"2024-04-24T18:10:43.444127+0000","last_peered":"2024-04-24T18:10:43.444127+0000","last_clean":"2024-04-24T18:10:43.444127+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:10:43.444127+0000","last_undegraded":"2024-04-24T18:10:43.444127+0000","last_fullsized":"2024-04-24T18:10:43.444127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449538+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:10:43.449538+0000","last_peered":"2024-04-24T18:10:43.449538+0000","last_clean":"2024-04-24T18:10:43.449538+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:10:43.449538+0000","last_undegraded":"2024-04-24T18:10:43.449538+0000","last_fullsized":"2024-04-24T18:10:43.449538+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":177,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:18.331936+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:31.195614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"174'957","reported_seq":7180,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444736+0000","last_change":"2024-04-24T18:10:21.336769+0000","last_active":"2024-04-24T18:10:43.444736+0000","last_peered":"2024-04-24T18:10:43.444736+0000","last_clean":"2024-04-24T18:10:43.444736+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:43.444736+0000","last_undegraded":"2024-04-24T18:10:43.444736+0000","last_fullsized":"2024-04-24T18:10:43.444736+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:53:36.139560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449814+0000","last_change":"2024-04-24T18:10:18.330231+0000","last_active":"2024-04-24T18:10:43.449814+0000","last_peered":"2024-04-24T18:10:43.449814+0000","last_clean":"2024-04-24T18:10:43.449814+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:43.449814+0000","last_undegraded":"2024-04-24T18:10:43.449814+0000","last_fullsized":"2024-04-24T18:10:43.449814+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:00:41.033763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":204,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450179+0000","last_change":"2024-04-24T18:10:18.328460+0000","last_active":"2024-04-24T18:10:43.450179+0000","last_peered":"2024-04-24T18:10:43.450179+0000","last_clean":"2024-04-24T18:10:43.450179+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:43.450179+0000","last_undegraded":"2024-04-24T18:10:43.450179+0000","last_fullsized":"2024-04-24T18:10:43.450179+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:04.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"165'1061","reported_seq":6785,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444268+0000","last_change":"2024-04-24T18:10:20.326388+0000","last_active":"2024-04-24T18:10:43.444268+0000","last_peered":"2024-04-24T18:10:43.444268+0000","last_clean":"2024-04-24T18:10:43.444268+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:43.444268+0000","last_undegraded":"2024-04-24T18:10:43.444268+0000","last_fullsized":"2024-04-24T18:10:43.444268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"159'4","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449661+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:10:43.449661+0000","last_peered":"2024-04-24T18:10:43.449661+0000","last_clean":"2024-04-24T18:10:43.449661+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:43.449661+0000","last_undegraded":"2024-04-24T18:10:43.449661+0000","last_fullsized":"2024-04-24T18:10:43.449661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450353+0000","last_change":"2024-04-24T18:10:18.331833+0000","last_active":"2024-04-24T18:10:43.450353+0000","last_peered":"2024-04-24T18:10:43.450353+0000","last_clean":"2024-04-24T18:10:43.450353+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:43.450353+0000","last_undegraded":"2024-04-24T18:10:43.450353+0000","last_fullsized":"2024-04-24T18:10:43.450353+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:47.993822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"168'981","reported_seq":10279,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449459+0000","last_change":"2024-04-24T18:10:20.327837+0000","last_active":"2024-04-24T18:10:43.449459+0000","last_peered":"2024-04-24T18:10:43.449459+0000","last_clean":"2024-04-24T18:10:43.449459+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:10:43.449459+0000","last_undegraded":"2024-04-24T18:10:43.449459+0000","last_fullsized":"2024-04-24T18:10:43.449459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444134+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:10:43.444134+0000","last_peered":"2024-04-24T18:10:43.444134+0000","last_clean":"2024-04-24T18:10:43.444134+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:10:43.444134+0000","last_undegraded":"2024-04-24T18:10:43.444134+0000","last_fullsized":"2024-04-24T18:10:43.444134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450351+0000","last_change":"2024-04-24T18:10:18.331400+0000","last_active":"2024-04-24T18:10:43.450351+0000","last_peered":"2024-04-24T18:10:43.450351+0000","last_clean":"2024-04-24T18:10:43.450351+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:10:43.450351+0000","last_undegraded":"2024-04-24T18:10:43.450351+0000","last_fullsized":"2024-04-24T18:10:43.450351+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:18.213539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"167'574","reported_seq":5342,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444387+0000","last_change":"2024-04-24T18:10:20.327045+0000","last_active":"2024-04-24T18:10:43.444387+0000","last_peered":"2024-04-24T18:10:43.444387+0000","last_clean":"2024-04-24T18:10:43.444387+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:43.444387+0000","last_undegraded":"2024-04-24T18:10:43.444387+0000","last_fullsized":"2024-04-24T18:10:43.444387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"113'2","reported_seq":40,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444509+0000","last_change":"2024-04-24T18:10:18.333879+0000","last_active":"2024-04-24T18:10:43.444509+0000","last_peered":"2024-04-24T18:10:43.444509+0000","last_clean":"2024-04-24T18:10:43.444509+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:43.444509+0000","last_undegraded":"2024-04-24T18:10:43.444509+0000","last_fullsized":"2024-04-24T18:10:43.444509+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:18.541992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":381,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450453+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:10:43.450453+0000","last_peered":"2024-04-24T18:10:43.450453+0000","last_clean":"2024-04-24T18:10:43.450453+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:10:43.450453+0000","last_undegraded":"2024-04-24T18:10:43.450453+0000","last_fullsized":"2024-04-24T18:10:43.450453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444276+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:10:43.444276+0000","last_peered":"2024-04-24T18:10:43.444276+0000","last_clean":"2024-04-24T18:10:43.444276+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:10:43.444276+0000","last_undegraded":"2024-04-24T18:10:43.444276+0000","last_fullsized":"2024-04-24T18:10:43.444276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"174'1090","reported_seq":10185,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445190+0000","last_change":"2024-04-24T18:10:22.338092+0000","last_active":"2024-04-24T18:10:43.445190+0000","last_peered":"2024-04-24T18:10:43.445190+0000","last_clean":"2024-04-24T18:10:43.445190+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:10:43.445190+0000","last_undegraded":"2024-04-24T18:10:43.445190+0000","last_fullsized":"2024-04-24T18:10:43.445190+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:06:01.070097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"174'671","reported_seq":6166,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449838+0000","last_change":"2024-04-24T18:10:20.332296+0000","last_active":"2024-04-24T18:10:43.449838+0000","last_peered":"2024-04-24T18:10:43.449838+0000","last_clean":"2024-04-24T18:10:43.449838+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:43.449838+0000","last_undegraded":"2024-04-24T18:10:43.449838+0000","last_fullsized":"2024-04-24T18:10:43.449838+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":183,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449716+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:10:43.449716+0000","last_peered":"2024-04-24T18:10:43.449716+0000","last_clean":"2024-04-24T18:10:43.449716+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:43.449716+0000","last_undegraded":"2024-04-24T18:10:43.449716+0000","last_fullsized":"2024-04-24T18:10:43.449716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"171'818","reported_seq":6149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329213+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":186,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449934+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:10:43.449934+0000","last_peered":"2024-04-24T18:10:43.449934+0000","last_clean":"2024-04-24T18:10:43.449934+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:43.449934+0000","last_undegraded":"2024-04-24T18:10:43.449934+0000","last_fullsized":"2024-04-24T18:10:43.449934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"165'696","reported_seq":12152,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450131+0000","last_change":"2024-04-24T18:10:20.328757+0000","last_active":"2024-04-24T18:10:43.450131+0000","last_peered":"2024-04-24T18:10:43.450131+0000","last_clean":"2024-04-24T18:10:43.450131+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:10:43.450131+0000","last_undegraded":"2024-04-24T18:10:43.450131+0000","last_fullsized":"2024-04-24T18:10:43.450131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"171'666","reported_seq":6306,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449742+0000","last_change":"2024-04-24T18:10:20.331718+0000","last_active":"2024-04-24T18:10:43.449742+0000","last_peered":"2024-04-24T18:10:43.449742+0000","last_clean":"2024-04-24T18:10:43.449742+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:43.449742+0000","last_undegraded":"2024-04-24T18:10:43.449742+0000","last_fullsized":"2024-04-24T18:10:43.449742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"167'694","reported_seq":3935,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:20.328410+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"171'682","reported_seq":6313,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444827+0000","last_change":"2024-04-24T18:10:21.332171+0000","last_active":"2024-04-24T18:10:43.444827+0000","last_peered":"2024-04-24T18:10:43.444827+0000","last_clean":"2024-04-24T18:10:43.444827+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:43.444827+0000","last_undegraded":"2024-04-24T18:10:43.444827+0000","last_fullsized":"2024-04-24T18:10:43.444827+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:41.461799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"171'580","reported_seq":7072,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449978+0000","last_change":"2024-04-24T18:10:20.332029+0000","last_active":"2024-04-24T18:10:43.449978+0000","last_peered":"2024-04-24T18:10:43.449978+0000","last_clean":"2024-04-24T18:10:43.449978+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:10:43.449978+0000","last_undegraded":"2024-04-24T18:10:43.449978+0000","last_fullsized":"2024-04-24T18:10:43.449978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"167'634","reported_seq":9280,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449993+0000","last_change":"2024-04-24T18:10:20.334138+0000","last_active":"2024-04-24T18:10:43.449993+0000","last_peered":"2024-04-24T18:10:43.449993+0000","last_clean":"2024-04-24T18:10:43.449993+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:43.449993+0000","last_undegraded":"2024-04-24T18:10:43.449993+0000","last_fullsized":"2024-04-24T18:10:43.449993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"167'596","reported_seq":7179,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449905+0000","last_change":"2024-04-24T18:10:20.334800+0000","last_active":"2024-04-24T18:10:43.449905+0000","last_peered":"2024-04-24T18:10:43.449905+0000","last_clean":"2024-04-24T18:10:43.449905+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:43.449905+0000","last_undegraded":"2024-04-24T18:10:43.449905+0000","last_fullsized":"2024-04-24T18:10:43.449905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"167'868","reported_seq":6431,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445558+0000","last_change":"2024-04-24T18:10:20.326786+0000","last_active":"2024-04-24T18:10:43.445558+0000","last_peered":"2024-04-24T18:10:43.445558+0000","last_clean":"2024-04-24T18:10:43.445558+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:10:43.445558+0000","last_undegraded":"2024-04-24T18:10:43.445558+0000","last_fullsized":"2024-04-24T18:10:43.445558+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"167'663","reported_seq":7482,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449450+0000","last_change":"2024-04-24T18:10:20.329149+0000","last_active":"2024-04-24T18:10:43.449450+0000","last_peered":"2024-04-24T18:10:43.449450+0000","last_clean":"2024-04-24T18:10:43.449450+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:43.449450+0000","last_undegraded":"2024-04-24T18:10:43.449450+0000","last_fullsized":"2024-04-24T18:10:43.449450+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"171'765","reported_seq":9197,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329631+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"174'662","reported_seq":6522,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444247+0000","last_change":"2024-04-24T18:10:20.325240+0000","last_active":"2024-04-24T18:10:43.444247+0000","last_peered":"2024-04-24T18:10:43.444247+0000","last_clean":"2024-04-24T18:10:43.444247+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:43.444247+0000","last_undegraded":"2024-04-24T18:10:43.444247+0000","last_fullsized":"2024-04-24T18:10:43.444247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"167'967","reported_seq":9149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449919+0000","last_change":"2024-04-24T18:10:20.334115+0000","last_active":"2024-04-24T18:10:43.449919+0000","last_peered":"2024-04-24T18:10:43.449919+0000","last_clean":"2024-04-24T18:10:43.449919+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:10:43.449919+0000","last_undegraded":"2024-04-24T18:10:43.449919+0000","last_fullsized":"2024-04-24T18:10:43.449919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"167'688","reported_seq":8040,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449785+0000","last_change":"2024-04-24T18:10:20.330052+0000","last_active":"2024-04-24T18:10:43.449785+0000","last_peered":"2024-04-24T18:10:43.449785+0000","last_clean":"2024-04-24T18:10:43.449785+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:10:43.449785+0000","last_undegraded":"2024-04-24T18:10:43.449785+0000","last_fullsized":"2024-04-24T18:10:43.449785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"167'621","reported_seq":6044,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450254+0000","last_change":"2024-04-24T18:10:20.329098+0000","last_active":"2024-04-24T18:10:43.450254+0000","last_peered":"2024-04-24T18:10:43.450254+0000","last_clean":"2024-04-24T18:10:43.450254+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:10:43.450254+0000","last_undegraded":"2024-04-24T18:10:43.450254+0000","last_fullsized":"2024-04-24T18:10:43.450254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"171'1163","reported_seq":7009,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445089+0000","last_change":"2024-04-24T18:10:22.337859+0000","last_active":"2024-04-24T18:10:43.445089+0000","last_peered":"2024-04-24T18:10:43.445089+0000","last_clean":"2024-04-24T18:10:43.445089+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:43.445089+0000","last_undegraded":"2024-04-24T18:10:43.445089+0000","last_fullsized":"2024-04-24T18:10:43.445089+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:55.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":26264,"ondisk_log_size":26264,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307912,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063928,"statfs":{"total":96636764160,"available":96321462272,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53000000000000003,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48599999999999999,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.67000000000000004,"5min":2.585,"15min":2.585},"last":0.64800000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78300000000000003,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":2.008,"15min":2.008},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.55100000000000005,"15min":0.51000000000000001},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.42999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.753,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":1.196,"15min":1.196},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":1.131,"15min":1.131},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77900000000000003,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75700000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893145,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309128,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":306784,"kb_avail":94062712,"statfs":{"total":96636764160,"available":96320217088,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":314146847},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":10.071999999999999,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.38}]},{"osd":2,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":2.9609999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.498},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307888,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063952,"statfs":{"total":96636764160,"available":96321486848,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.53500000000000003,"15min":0.61799999999999999},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.5,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.89500000000000002,"15min":1.127},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":14.15,"15min":14.15},"last":0.28199999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.93600000000000005,"5min":1.306,"15min":1.306},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.96099999999999997,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.95499999999999996,"5min":14.118,"15min":14.118},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.93000000000000005,"15min":1.1519999999999999},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67600000000000005,"5min":14.089,"15min":14.089},"last":0.371},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.313}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":0.61099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71799999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.0600000000000001,"15min":1.274},"min":{"1min":0.35699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.76300000000000001},"min":{"1min":0.371,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2310000000000001,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.746,"5min":1.49,"15min":1.49},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.073,"15min":1.2929999999999999},"min":{"1min":0.438,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1279999999999999,"5min":14.051,"15min":14.051},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.74399999999999999,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.1870000000000001,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311956,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059884,"statfs":{"total":96636764160,"available":96317321216,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65700000000000003,"5min":1.375,"15min":1.375},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":1.2549999999999999,"15min":1.6379999999999999},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90200000000000002,"5min":26.936,"15min":26.936},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.92000000000000004,"15min":1.1479999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":11.93,"15min":11.93},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42699999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.157,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.83599999999999997,"15min":0.995},"min":{"1min":0.25900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.93799999999999994,"5min":14.371,"15min":14.371},"last":0.432},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.86699999999999999,"15min":0.97399999999999998},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.92800000000000005,"15min":1.1140000000000001},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":14.414,"15min":14.414},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.93200000000000005,"5min":1.329,"15min":1.329},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83099999999999996,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97099999999999997,"15min":1.133},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.995,"5min":14.452,"15min":14.452},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86099999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.98699999999999999,"15min":1.1479999999999999},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1259999999999999,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299656,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072184,"statfs":{"total":96636764160,"available":96329916416,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":4.109,"15min":4.109},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.441,"15min":0.42099999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77900000000000003,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.434,"15min":0.40699999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":1.619,"15min":1.619},"last":0.23699999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.316}]},{"osd":4,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":1.155,"15min":1.155},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.71799999999999997,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":10.516,"15min":10.516},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.56999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303764,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2367488,"data_stored":1201291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36299999999999999,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.41799999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28499999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48099999999999998,"5min":1.792,"15min":1.792},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.44800000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.50900000000000001,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.436}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68400000000000005,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.76300000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70699999999999996,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.377}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.755,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.74299999999999999,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":15.4,"15min":15.4},"last":0.53200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311952,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059888,"statfs":{"total":96636764160,"available":96317325312,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46800000000000003,"15min":0.47499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.5},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.497},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55000000000000004,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.626,"5min":0.627,"15min":0.627},"last":0.371}]},{"osd":2,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.33900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.27600000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63800000000000001,"5min":1.153,"15min":1.153},"last":0.40500000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":1.373,"15min":1.373},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71099999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.60999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2363392,"data_stored":1201272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.307,"5min":0.189,"15min":0.189},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":3.5840000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":3.7130000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":1.296,"15min":1.296},"last":1.407},{"interface":"front","average":{"1min":0.44,"5min":0.504,"15min":0.53100000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63300000000000001,"5min":1.377,"15min":1.377},"last":1.361}]},{"osd":3,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":1.899,"15min":1.899},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59899999999999998,"5min":1.881,"15min":1.881},"last":0.997}]},{"osd":4,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46999999999999997,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.66100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.376},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52000000000000002,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.432}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":1.016,"15min":1.016},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54400000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:10:49.713 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:10:50.045 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:50.047 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:10:50.058 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-24T18:10:49.337839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590640,"num_objects":1800,"num_object_clones":0,"num_object_copies":3600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29264,"num_write_kb":534157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26387,"ondisk_log_size":26387,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456016,"kb_used_data":18528,"kb_used_omap":43,"kb_used_meta":2437332,"kb_avail":752518704,"statfs":{"total":773094113280,"available":770579152896,"internally_reserved":0,"allocated":18972672,"data_stored":10790096,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2495828359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.900840"},"pg_stats":[{"pgid":"4.1f","version":"173'683","reported_seq":7230,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449490+0000","last_change":"2024-04-24T18:10:27.328087+0000","last_active":"2024-04-24T18:10:43.449490+0000","last_peered":"2024-04-24T18:10:43.449490+0000","last_clean":"2024-04-24T18:10:43.449490+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:10:43.449490+0000","last_undegraded":"2024-04-24T18:10:43.449490+0000","last_fullsized":"2024-04-24T18:10:43.449490+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:17.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"160'1339","reported_seq":7257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445000+0000","last_change":"2024-04-24T18:10:21.335419+0000","last_active":"2024-04-24T18:10:43.445000+0000","last_peered":"2024-04-24T18:10:43.445000+0000","last_clean":"2024-04-24T18:10:43.445000+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:43.445000+0000","last_undegraded":"2024-04-24T18:10:43.445000+0000","last_fullsized":"2024-04-24T18:10:43.445000+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1339,"log_dups_size":0,"ondisk_log_size":1339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:03.319026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"167'710","reported_seq":5997,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449714+0000","last_change":"2024-04-24T18:10:20.330289+0000","last_active":"2024-04-24T18:10:43.449714+0000","last_peered":"2024-04-24T18:10:43.449714+0000","last_clean":"2024-04-24T18:10:43.449714+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:10:43.449714+0000","last_undegraded":"2024-04-24T18:10:43.449714+0000","last_fullsized":"2024-04-24T18:10:43.449714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1079","reported_seq":10264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445678+0000","last_change":"2024-04-24T18:10:20.345289+0000","last_active":"2024-04-24T18:10:43.445678+0000","last_peered":"2024-04-24T18:10:43.445678+0000","last_clean":"2024-04-24T18:10:43.445678+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:43.445678+0000","last_undegraded":"2024-04-24T18:10:43.445678+0000","last_fullsized":"2024-04-24T18:10:43.445678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"171'535","reported_seq":11656,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449807+0000","last_change":"2024-04-24T18:10:20.332101+0000","last_active":"2024-04-24T18:10:43.449807+0000","last_peered":"2024-04-24T18:10:43.449807+0000","last_clean":"2024-04-24T18:10:43.449807+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:43.449807+0000","last_undegraded":"2024-04-24T18:10:43.449807+0000","last_fullsized":"2024-04-24T18:10:43.449807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"167'880","reported_seq":12337,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449756+0000","last_change":"2024-04-24T18:10:20.331679+0000","last_active":"2024-04-24T18:10:43.449756+0000","last_peered":"2024-04-24T18:10:43.449756+0000","last_clean":"2024-04-24T18:10:43.449756+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:43.449756+0000","last_undegraded":"2024-04-24T18:10:43.449756+0000","last_fullsized":"2024-04-24T18:10:43.449756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"167'726","reported_seq":9735,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449505+0000","last_change":"2024-04-24T18:10:20.329954+0000","last_active":"2024-04-24T18:10:43.449505+0000","last_peered":"2024-04-24T18:10:43.449505+0000","last_clean":"2024-04-24T18:10:43.449505+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:43.449505+0000","last_undegraded":"2024-04-24T18:10:43.449505+0000","last_fullsized":"2024-04-24T18:10:43.449505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"167'616","reported_seq":11663,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449727+0000","last_change":"2024-04-24T18:10:20.334440+0000","last_active":"2024-04-24T18:10:43.449727+0000","last_peered":"2024-04-24T18:10:43.449727+0000","last_clean":"2024-04-24T18:10:43.449727+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:10:43.449727+0000","last_undegraded":"2024-04-24T18:10:43.449727+0000","last_fullsized":"2024-04-24T18:10:43.449727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"167'1362","reported_seq":6969,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449833+0000","last_change":"2024-04-24T18:10:20.329528+0000","last_active":"2024-04-24T18:10:43.449833+0000","last_peered":"2024-04-24T18:10:43.449833+0000","last_clean":"2024-04-24T18:10:43.449833+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:10:43.449833+0000","last_undegraded":"2024-04-24T18:10:43.449833+0000","last_fullsized":"2024-04-24T18:10:43.449833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"174'1237","reported_seq":4971,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444127+0000","last_change":"2024-04-24T18:10:20.324905+0000","last_active":"2024-04-24T18:10:43.444127+0000","last_peered":"2024-04-24T18:10:43.444127+0000","last_clean":"2024-04-24T18:10:43.444127+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:10:43.444127+0000","last_undegraded":"2024-04-24T18:10:43.444127+0000","last_fullsized":"2024-04-24T18:10:43.444127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449538+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:10:43.449538+0000","last_peered":"2024-04-24T18:10:43.449538+0000","last_clean":"2024-04-24T18:10:43.449538+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:10:43.449538+0000","last_undegraded":"2024-04-24T18:10:43.449538+0000","last_fullsized":"2024-04-24T18:10:43.449538+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":177,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:18.331936+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:31.195614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"174'957","reported_seq":7180,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444736+0000","last_change":"2024-04-24T18:10:21.336769+0000","last_active":"2024-04-24T18:10:43.444736+0000","last_peered":"2024-04-24T18:10:43.444736+0000","last_clean":"2024-04-24T18:10:43.444736+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:43.444736+0000","last_undegraded":"2024-04-24T18:10:43.444736+0000","last_fullsized":"2024-04-24T18:10:43.444736+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:53:36.139560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449814+0000","last_change":"2024-04-24T18:10:18.330231+0000","last_active":"2024-04-24T18:10:43.449814+0000","last_peered":"2024-04-24T18:10:43.449814+0000","last_clean":"2024-04-24T18:10:43.449814+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:43.449814+0000","last_undegraded":"2024-04-24T18:10:43.449814+0000","last_fullsized":"2024-04-24T18:10:43.449814+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:00:41.033763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":204,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450179+0000","last_change":"2024-04-24T18:10:18.328460+0000","last_active":"2024-04-24T18:10:43.450179+0000","last_peered":"2024-04-24T18:10:43.450179+0000","last_clean":"2024-04-24T18:10:43.450179+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:43.450179+0000","last_undegraded":"2024-04-24T18:10:43.450179+0000","last_fullsized":"2024-04-24T18:10:43.450179+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:04.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"165'1061","reported_seq":6785,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444268+0000","last_change":"2024-04-24T18:10:20.326388+0000","last_active":"2024-04-24T18:10:43.444268+0000","last_peered":"2024-04-24T18:10:43.444268+0000","last_clean":"2024-04-24T18:10:43.444268+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:43.444268+0000","last_undegraded":"2024-04-24T18:10:43.444268+0000","last_fullsized":"2024-04-24T18:10:43.444268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"159'4","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449661+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:10:43.449661+0000","last_peered":"2024-04-24T18:10:43.449661+0000","last_clean":"2024-04-24T18:10:43.449661+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:43.449661+0000","last_undegraded":"2024-04-24T18:10:43.449661+0000","last_fullsized":"2024-04-24T18:10:43.449661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450353+0000","last_change":"2024-04-24T18:10:18.331833+0000","last_active":"2024-04-24T18:10:43.450353+0000","last_peered":"2024-04-24T18:10:43.450353+0000","last_clean":"2024-04-24T18:10:43.450353+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:43.450353+0000","last_undegraded":"2024-04-24T18:10:43.450353+0000","last_fullsized":"2024-04-24T18:10:43.450353+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:47.993822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"168'981","reported_seq":10279,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449459+0000","last_change":"2024-04-24T18:10:20.327837+0000","last_active":"2024-04-24T18:10:43.449459+0000","last_peered":"2024-04-24T18:10:43.449459+0000","last_clean":"2024-04-24T18:10:43.449459+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:10:43.449459+0000","last_undegraded":"2024-04-24T18:10:43.449459+0000","last_fullsized":"2024-04-24T18:10:43.449459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444134+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:10:43.444134+0000","last_peered":"2024-04-24T18:10:43.444134+0000","last_clean":"2024-04-24T18:10:43.444134+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:10:43.444134+0000","last_undegraded":"2024-04-24T18:10:43.444134+0000","last_fullsized":"2024-04-24T18:10:43.444134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450351+0000","last_change":"2024-04-24T18:10:18.331400+0000","last_active":"2024-04-24T18:10:43.450351+0000","last_peered":"2024-04-24T18:10:43.450351+0000","last_clean":"2024-04-24T18:10:43.450351+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:10:43.450351+0000","last_undegraded":"2024-04-24T18:10:43.450351+0000","last_fullsized":"2024-04-24T18:10:43.450351+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:18.213539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"167'574","reported_seq":5342,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444387+0000","last_change":"2024-04-24T18:10:20.327045+0000","last_active":"2024-04-24T18:10:43.444387+0000","last_peered":"2024-04-24T18:10:43.444387+0000","last_clean":"2024-04-24T18:10:43.444387+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:43.444387+0000","last_undegraded":"2024-04-24T18:10:43.444387+0000","last_fullsized":"2024-04-24T18:10:43.444387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"113'2","reported_seq":40,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444509+0000","last_change":"2024-04-24T18:10:18.333879+0000","last_active":"2024-04-24T18:10:43.444509+0000","last_peered":"2024-04-24T18:10:43.444509+0000","last_clean":"2024-04-24T18:10:43.444509+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:43.444509+0000","last_undegraded":"2024-04-24T18:10:43.444509+0000","last_fullsized":"2024-04-24T18:10:43.444509+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:18.541992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":381,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450453+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:10:43.450453+0000","last_peered":"2024-04-24T18:10:43.450453+0000","last_clean":"2024-04-24T18:10:43.450453+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:10:43.450453+0000","last_undegraded":"2024-04-24T18:10:43.450453+0000","last_fullsized":"2024-04-24T18:10:43.450453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444276+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:10:43.444276+0000","last_peered":"2024-04-24T18:10:43.444276+0000","last_clean":"2024-04-24T18:10:43.444276+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:10:43.444276+0000","last_undegraded":"2024-04-24T18:10:43.444276+0000","last_fullsized":"2024-04-24T18:10:43.444276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"174'1090","reported_seq":10185,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445190+0000","last_change":"2024-04-24T18:10:22.338092+0000","last_active":"2024-04-24T18:10:43.445190+0000","last_peered":"2024-04-24T18:10:43.445190+0000","last_clean":"2024-04-24T18:10:43.445190+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:10:43.445190+0000","last_undegraded":"2024-04-24T18:10:43.445190+0000","last_fullsized":"2024-04-24T18:10:43.445190+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:06:01.070097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"174'671","reported_seq":6166,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449838+0000","last_change":"2024-04-24T18:10:20.332296+0000","last_active":"2024-04-24T18:10:43.449838+0000","last_peered":"2024-04-24T18:10:43.449838+0000","last_clean":"2024-04-24T18:10:43.449838+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:43.449838+0000","last_undegraded":"2024-04-24T18:10:43.449838+0000","last_fullsized":"2024-04-24T18:10:43.449838+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":183,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449716+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:10:43.449716+0000","last_peered":"2024-04-24T18:10:43.449716+0000","last_clean":"2024-04-24T18:10:43.449716+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:43.449716+0000","last_undegraded":"2024-04-24T18:10:43.449716+0000","last_fullsized":"2024-04-24T18:10:43.449716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"171'818","reported_seq":6149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329213+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":186,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449934+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:10:43.449934+0000","last_peered":"2024-04-24T18:10:43.449934+0000","last_clean":"2024-04-24T18:10:43.449934+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:43.449934+0000","last_undegraded":"2024-04-24T18:10:43.449934+0000","last_fullsized":"2024-04-24T18:10:43.449934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"165'696","reported_seq":12152,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450131+0000","last_change":"2024-04-24T18:10:20.328757+0000","last_active":"2024-04-24T18:10:43.450131+0000","last_peered":"2024-04-24T18:10:43.450131+0000","last_clean":"2024-04-24T18:10:43.450131+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:10:43.450131+0000","last_undegraded":"2024-04-24T18:10:43.450131+0000","last_fullsized":"2024-04-24T18:10:43.450131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"171'666","reported_seq":6306,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449742+0000","last_change":"2024-04-24T18:10:20.331718+0000","last_active":"2024-04-24T18:10:43.449742+0000","last_peered":"2024-04-24T18:10:43.449742+0000","last_clean":"2024-04-24T18:10:43.449742+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:43.449742+0000","last_undegraded":"2024-04-24T18:10:43.449742+0000","last_fullsized":"2024-04-24T18:10:43.449742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"167'694","reported_seq":3935,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:20.328410+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"171'682","reported_seq":6313,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444827+0000","last_change":"2024-04-24T18:10:21.332171+0000","last_active":"2024-04-24T18:10:43.444827+0000","last_peered":"2024-04-24T18:10:43.444827+0000","last_clean":"2024-04-24T18:10:43.444827+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:43.444827+0000","last_undegraded":"2024-04-24T18:10:43.444827+0000","last_fullsized":"2024-04-24T18:10:43.444827+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:41.461799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"171'580","reported_seq":7072,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449978+0000","last_change":"2024-04-24T18:10:20.332029+0000","last_active":"2024-04-24T18:10:43.449978+0000","last_peered":"2024-04-24T18:10:43.449978+0000","last_clean":"2024-04-24T18:10:43.449978+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:10:43.449978+0000","last_undegraded":"2024-04-24T18:10:43.449978+0000","last_fullsized":"2024-04-24T18:10:43.449978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"167'634","reported_seq":9280,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449993+0000","last_change":"2024-04-24T18:10:20.334138+0000","last_active":"2024-04-24T18:10:43.449993+0000","last_peered":"2024-04-24T18:10:43.449993+0000","last_clean":"2024-04-24T18:10:43.449993+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:43.449993+0000","last_undegraded":"2024-04-24T18:10:43.449993+0000","last_fullsized":"2024-04-24T18:10:43.449993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"167'596","reported_seq":7179,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449905+0000","last_change":"2024-04-24T18:10:20.334800+0000","last_active":"2024-04-24T18:10:43.449905+0000","last_peered":"2024-04-24T18:10:43.449905+0000","last_clean":"2024-04-24T18:10:43.449905+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:43.449905+0000","last_undegraded":"2024-04-24T18:10:43.449905+0000","last_fullsized":"2024-04-24T18:10:43.449905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"167'868","reported_seq":6431,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445558+0000","last_change":"2024-04-24T18:10:20.326786+0000","last_active":"2024-04-24T18:10:43.445558+0000","last_peered":"2024-04-24T18:10:43.445558+0000","last_clean":"2024-04-24T18:10:43.445558+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:10:43.445558+0000","last_undegraded":"2024-04-24T18:10:43.445558+0000","last_fullsized":"2024-04-24T18:10:43.445558+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"167'663","reported_seq":7482,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449450+0000","last_change":"2024-04-24T18:10:20.329149+0000","last_active":"2024-04-24T18:10:43.449450+0000","last_peered":"2024-04-24T18:10:43.449450+0000","last_clean":"2024-04-24T18:10:43.449450+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:43.449450+0000","last_undegraded":"2024-04-24T18:10:43.449450+0000","last_fullsized":"2024-04-24T18:10:43.449450+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"171'765","reported_seq":9197,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329631+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"174'662","reported_seq":6522,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444247+0000","last_change":"2024-04-24T18:10:20.325240+0000","last_active":"2024-04-24T18:10:43.444247+0000","last_peered":"2024-04-24T18:10:43.444247+0000","last_clean":"2024-04-24T18:10:43.444247+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:43.444247+0000","last_undegraded":"2024-04-24T18:10:43.444247+0000","last_fullsized":"2024-04-24T18:10:43.444247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"167'967","reported_seq":9149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449919+0000","last_change":"2024-04-24T18:10:20.334115+0000","last_active":"2024-04-24T18:10:43.449919+0000","last_peered":"2024-04-24T18:10:43.449919+0000","last_clean":"2024-04-24T18:10:43.449919+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:10:43.449919+0000","last_undegraded":"2024-04-24T18:10:43.449919+0000","last_fullsized":"2024-04-24T18:10:43.449919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"167'688","reported_seq":8040,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449785+0000","last_change":"2024-04-24T18:10:20.330052+0000","last_active":"2024-04-24T18:10:43.449785+0000","last_peered":"2024-04-24T18:10:43.449785+0000","last_clean":"2024-04-24T18:10:43.449785+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:10:43.449785+0000","last_undegraded":"2024-04-24T18:10:43.449785+0000","last_fullsized":"2024-04-24T18:10:43.449785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"167'621","reported_seq":6044,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450254+0000","last_change":"2024-04-24T18:10:20.329098+0000","last_active":"2024-04-24T18:10:43.450254+0000","last_peered":"2024-04-24T18:10:43.450254+0000","last_clean":"2024-04-24T18:10:43.450254+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:10:43.450254+0000","last_undegraded":"2024-04-24T18:10:43.450254+0000","last_fullsized":"2024-04-24T18:10:43.450254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"171'1163","reported_seq":7009,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445089+0000","last_change":"2024-04-24T18:10:22.337859+0000","last_active":"2024-04-24T18:10:43.445089+0000","last_peered":"2024-04-24T18:10:43.445089+0000","last_clean":"2024-04-24T18:10:43.445089+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:43.445089+0000","last_undegraded":"2024-04-24T18:10:43.445089+0000","last_fullsized":"2024-04-24T18:10:43.445089+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:55.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":26264,"ondisk_log_size":26264,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307912,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063928,"statfs":{"total":96636764160,"available":96321462272,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53000000000000003,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48599999999999999,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.67000000000000004,"5min":2.585,"15min":2.585},"last":0.64800000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78300000000000003,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":2.008,"15min":2.008},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.55100000000000005,"15min":0.51000000000000001},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.42999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.753,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":1.196,"15min":1.196},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":1.131,"15min":1.131},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77900000000000003,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75700000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893145,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309128,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":306784,"kb_avail":94062712,"statfs":{"total":96636764160,"available":96320217088,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":314146847},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":10.071999999999999,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.38}]},{"osd":2,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":2.9609999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.498},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307888,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063952,"statfs":{"total":96636764160,"available":96321486848,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.53500000000000003,"15min":0.61799999999999999},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.5,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.89500000000000002,"15min":1.127},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":14.15,"15min":14.15},"last":0.28199999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.93600000000000005,"5min":1.306,"15min":1.306},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.96099999999999997,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.95499999999999996,"5min":14.118,"15min":14.118},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.93000000000000005,"15min":1.1519999999999999},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67600000000000005,"5min":14.089,"15min":14.089},"last":0.371},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.313}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":0.61099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71799999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.0600000000000001,"15min":1.274},"min":{"1min":0.35699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.76300000000000001},"min":{"1min":0.371,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2310000000000001,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.746,"5min":1.49,"15min":1.49},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.073,"15min":1.2929999999999999},"min":{"1min":0.438,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1279999999999999,"5min":14.051,"15min":14.051},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.74399999999999999,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.1870000000000001,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311956,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059884,"statfs":{"total":96636764160,"available":96317321216,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65700000000000003,"5min":1.375,"15min":1.375},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":1.2549999999999999,"15min":1.6379999999999999},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90200000000000002,"5min":26.936,"15min":26.936},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.92000000000000004,"15min":1.1479999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":11.93,"15min":11.93},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42699999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.157,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.83599999999999997,"15min":0.995},"min":{"1min":0.25900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.93799999999999994,"5min":14.371,"15min":14.371},"last":0.432},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.86699999999999999,"15min":0.97399999999999998},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.92800000000000005,"15min":1.1140000000000001},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":14.414,"15min":14.414},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.93200000000000005,"5min":1.329,"15min":1.329},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83099999999999996,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97099999999999997,"15min":1.133},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.995,"5min":14.452,"15min":14.452},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86099999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.98699999999999999,"15min":1.1479999999999999},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1259999999999999,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299656,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072184,"statfs":{"total":96636764160,"available":96329916416,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":4.109,"15min":4.109},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.441,"15min":0.42099999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77900000000000003,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.434,"15min":0.40699999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":1.619,"15min":1.619},"last":0.23699999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.316}]},{"osd":4,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":1.155,"15min":1.155},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.71799999999999997,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":10.516,"15min":10.516},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.56999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303764,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2367488,"data_stored":1201291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36299999999999999,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.41799999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28499999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48099999999999998,"5min":1.792,"15min":1.792},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.44800000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.50900000000000001,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.436}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68400000000000005,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.76300000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70699999999999996,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.377}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.755,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.74299999999999999,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":15.4,"15min":15.4},"last":0.53200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311952,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059888,"statfs":{"total":96636764160,"available":96317325312,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46800000000000003,"15min":0.47499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.5},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.497},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55000000000000004,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.626,"5min":0.627,"15min":0.627},"last":0.371}]},{"osd":2,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.33900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.27600000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63800000000000001,"5min":1.153,"15min":1.153},"last":0.40500000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":1.373,"15min":1.373},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71099999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.60999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2363392,"data_stored":1201272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.307,"5min":0.189,"15min":0.189},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":3.5840000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":3.7130000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":1.296,"15min":1.296},"last":1.407},{"interface":"front","average":{"1min":0.44,"5min":0.504,"15min":0.53100000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63300000000000001,"5min":1.377,"15min":1.377},"last":1.361}]},{"osd":3,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":1.899,"15min":1.899},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59899999999999998,"5min":1.881,"15min":1.881},"last":0.997}]},{"osd":4,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46999999999999997,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.66100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.376},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52000000000000002,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.432}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":1.016,"15min":1.016},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54400000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:10:50.061 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-24T18:10:50.061 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:10:50.394 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:50.395 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:10:50.407 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-04-24T18:10:49.337839+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590640,"num_objects":1800,"num_object_clones":0,"num_object_copies":3600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29264,"num_write_kb":534157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26387,"ondisk_log_size":26387,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456016,"kb_used_data":18528,"kb_used_omap":43,"kb_used_meta":2437332,"kb_avail":752518704,"statfs":{"total":773094113280,"available":770579152896,"internally_reserved":0,"allocated":18972672,"data_stored":10790096,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2495828359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.900840"},"pg_stats":[{"pgid":"4.1f","version":"173'683","reported_seq":7230,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449490+0000","last_change":"2024-04-24T18:10:27.328087+0000","last_active":"2024-04-24T18:10:43.449490+0000","last_peered":"2024-04-24T18:10:43.449490+0000","last_clean":"2024-04-24T18:10:43.449490+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:10:43.449490+0000","last_undegraded":"2024-04-24T18:10:43.449490+0000","last_fullsized":"2024-04-24T18:10:43.449490+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:17.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"160'1339","reported_seq":7257,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445000+0000","last_change":"2024-04-24T18:10:21.335419+0000","last_active":"2024-04-24T18:10:43.445000+0000","last_peered":"2024-04-24T18:10:43.445000+0000","last_clean":"2024-04-24T18:10:43.445000+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:43.445000+0000","last_undegraded":"2024-04-24T18:10:43.445000+0000","last_fullsized":"2024-04-24T18:10:43.445000+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1339,"log_dups_size":0,"ondisk_log_size":1339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:17:03.319026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"167'710","reported_seq":5997,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449714+0000","last_change":"2024-04-24T18:10:20.330289+0000","last_active":"2024-04-24T18:10:43.449714+0000","last_peered":"2024-04-24T18:10:43.449714+0000","last_clean":"2024-04-24T18:10:43.449714+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:10:43.449714+0000","last_undegraded":"2024-04-24T18:10:43.449714+0000","last_fullsized":"2024-04-24T18:10:43.449714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1079","reported_seq":10264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445678+0000","last_change":"2024-04-24T18:10:20.345289+0000","last_active":"2024-04-24T18:10:43.445678+0000","last_peered":"2024-04-24T18:10:43.445678+0000","last_clean":"2024-04-24T18:10:43.445678+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:43.445678+0000","last_undegraded":"2024-04-24T18:10:43.445678+0000","last_fullsized":"2024-04-24T18:10:43.445678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"171'535","reported_seq":11656,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449807+0000","last_change":"2024-04-24T18:10:20.332101+0000","last_active":"2024-04-24T18:10:43.449807+0000","last_peered":"2024-04-24T18:10:43.449807+0000","last_clean":"2024-04-24T18:10:43.449807+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:43.449807+0000","last_undegraded":"2024-04-24T18:10:43.449807+0000","last_fullsized":"2024-04-24T18:10:43.449807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"167'880","reported_seq":12337,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449756+0000","last_change":"2024-04-24T18:10:20.331679+0000","last_active":"2024-04-24T18:10:43.449756+0000","last_peered":"2024-04-24T18:10:43.449756+0000","last_clean":"2024-04-24T18:10:43.449756+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:43.449756+0000","last_undegraded":"2024-04-24T18:10:43.449756+0000","last_fullsized":"2024-04-24T18:10:43.449756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"167'726","reported_seq":9735,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449505+0000","last_change":"2024-04-24T18:10:20.329954+0000","last_active":"2024-04-24T18:10:43.449505+0000","last_peered":"2024-04-24T18:10:43.449505+0000","last_clean":"2024-04-24T18:10:43.449505+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:43.449505+0000","last_undegraded":"2024-04-24T18:10:43.449505+0000","last_fullsized":"2024-04-24T18:10:43.449505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"167'616","reported_seq":11663,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449727+0000","last_change":"2024-04-24T18:10:20.334440+0000","last_active":"2024-04-24T18:10:43.449727+0000","last_peered":"2024-04-24T18:10:43.449727+0000","last_clean":"2024-04-24T18:10:43.449727+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:10:43.449727+0000","last_undegraded":"2024-04-24T18:10:43.449727+0000","last_fullsized":"2024-04-24T18:10:43.449727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"167'1362","reported_seq":6969,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449833+0000","last_change":"2024-04-24T18:10:20.329528+0000","last_active":"2024-04-24T18:10:43.449833+0000","last_peered":"2024-04-24T18:10:43.449833+0000","last_clean":"2024-04-24T18:10:43.449833+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:10:43.449833+0000","last_undegraded":"2024-04-24T18:10:43.449833+0000","last_fullsized":"2024-04-24T18:10:43.449833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"174'1237","reported_seq":4971,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444127+0000","last_change":"2024-04-24T18:10:20.324905+0000","last_active":"2024-04-24T18:10:43.444127+0000","last_peered":"2024-04-24T18:10:43.444127+0000","last_clean":"2024-04-24T18:10:43.444127+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:10:43.444127+0000","last_undegraded":"2024-04-24T18:10:43.444127+0000","last_fullsized":"2024-04-24T18:10:43.444127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449538+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:10:43.449538+0000","last_peered":"2024-04-24T18:10:43.449538+0000","last_clean":"2024-04-24T18:10:43.449538+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:10:43.449538+0000","last_undegraded":"2024-04-24T18:10:43.449538+0000","last_fullsized":"2024-04-24T18:10:43.449538+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":177,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:18.331936+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:31.195614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"174'957","reported_seq":7180,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444736+0000","last_change":"2024-04-24T18:10:21.336769+0000","last_active":"2024-04-24T18:10:43.444736+0000","last_peered":"2024-04-24T18:10:43.444736+0000","last_clean":"2024-04-24T18:10:43.444736+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:43.444736+0000","last_undegraded":"2024-04-24T18:10:43.444736+0000","last_fullsized":"2024-04-24T18:10:43.444736+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:53:36.139560+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449814+0000","last_change":"2024-04-24T18:10:18.330231+0000","last_active":"2024-04-24T18:10:43.449814+0000","last_peered":"2024-04-24T18:10:43.449814+0000","last_clean":"2024-04-24T18:10:43.449814+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:43.449814+0000","last_undegraded":"2024-04-24T18:10:43.449814+0000","last_fullsized":"2024-04-24T18:10:43.449814+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:00:41.033763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":204,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450179+0000","last_change":"2024-04-24T18:10:18.328460+0000","last_active":"2024-04-24T18:10:43.450179+0000","last_peered":"2024-04-24T18:10:43.450179+0000","last_clean":"2024-04-24T18:10:43.450179+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:43.450179+0000","last_undegraded":"2024-04-24T18:10:43.450179+0000","last_fullsized":"2024-04-24T18:10:43.450179+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:04.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"165'1061","reported_seq":6785,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444268+0000","last_change":"2024-04-24T18:10:20.326388+0000","last_active":"2024-04-24T18:10:43.444268+0000","last_peered":"2024-04-24T18:10:43.444268+0000","last_clean":"2024-04-24T18:10:43.444268+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:43.444268+0000","last_undegraded":"2024-04-24T18:10:43.444268+0000","last_fullsized":"2024-04-24T18:10:43.444268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"159'4","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449661+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:10:43.449661+0000","last_peered":"2024-04-24T18:10:43.449661+0000","last_clean":"2024-04-24T18:10:43.449661+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:43.449661+0000","last_undegraded":"2024-04-24T18:10:43.449661+0000","last_fullsized":"2024-04-24T18:10:43.449661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450353+0000","last_change":"2024-04-24T18:10:18.331833+0000","last_active":"2024-04-24T18:10:43.450353+0000","last_peered":"2024-04-24T18:10:43.450353+0000","last_clean":"2024-04-24T18:10:43.450353+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:43.450353+0000","last_undegraded":"2024-04-24T18:10:43.450353+0000","last_fullsized":"2024-04-24T18:10:43.450353+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:47.993822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"168'981","reported_seq":10279,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449459+0000","last_change":"2024-04-24T18:10:20.327837+0000","last_active":"2024-04-24T18:10:43.449459+0000","last_peered":"2024-04-24T18:10:43.449459+0000","last_clean":"2024-04-24T18:10:43.449459+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:10:43.449459+0000","last_undegraded":"2024-04-24T18:10:43.449459+0000","last_fullsized":"2024-04-24T18:10:43.449459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444134+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:10:43.444134+0000","last_peered":"2024-04-24T18:10:43.444134+0000","last_clean":"2024-04-24T18:10:43.444134+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:10:43.444134+0000","last_undegraded":"2024-04-24T18:10:43.444134+0000","last_fullsized":"2024-04-24T18:10:43.444134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450351+0000","last_change":"2024-04-24T18:10:18.331400+0000","last_active":"2024-04-24T18:10:43.450351+0000","last_peered":"2024-04-24T18:10:43.450351+0000","last_clean":"2024-04-24T18:10:43.450351+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:10:43.450351+0000","last_undegraded":"2024-04-24T18:10:43.450351+0000","last_fullsized":"2024-04-24T18:10:43.450351+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:18.213539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"167'574","reported_seq":5342,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444387+0000","last_change":"2024-04-24T18:10:20.327045+0000","last_active":"2024-04-24T18:10:43.444387+0000","last_peered":"2024-04-24T18:10:43.444387+0000","last_clean":"2024-04-24T18:10:43.444387+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:43.444387+0000","last_undegraded":"2024-04-24T18:10:43.444387+0000","last_fullsized":"2024-04-24T18:10:43.444387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"113'2","reported_seq":40,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444509+0000","last_change":"2024-04-24T18:10:18.333879+0000","last_active":"2024-04-24T18:10:43.444509+0000","last_peered":"2024-04-24T18:10:43.444509+0000","last_clean":"2024-04-24T18:10:43.444509+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:43.444509+0000","last_undegraded":"2024-04-24T18:10:43.444509+0000","last_fullsized":"2024-04-24T18:10:43.444509+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:24:18.541992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":381,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450453+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:10:43.450453+0000","last_peered":"2024-04-24T18:10:43.450453+0000","last_clean":"2024-04-24T18:10:43.450453+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:10:43.450453+0000","last_undegraded":"2024-04-24T18:10:43.450453+0000","last_fullsized":"2024-04-24T18:10:43.450453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444276+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:10:43.444276+0000","last_peered":"2024-04-24T18:10:43.444276+0000","last_clean":"2024-04-24T18:10:43.444276+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:10:43.444276+0000","last_undegraded":"2024-04-24T18:10:43.444276+0000","last_fullsized":"2024-04-24T18:10:43.444276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"174'1090","reported_seq":10185,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445190+0000","last_change":"2024-04-24T18:10:22.338092+0000","last_active":"2024-04-24T18:10:43.445190+0000","last_peered":"2024-04-24T18:10:43.445190+0000","last_clean":"2024-04-24T18:10:43.445190+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:10:43.445190+0000","last_undegraded":"2024-04-24T18:10:43.445190+0000","last_fullsized":"2024-04-24T18:10:43.445190+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:06:01.070097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"174'671","reported_seq":6166,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449838+0000","last_change":"2024-04-24T18:10:20.332296+0000","last_active":"2024-04-24T18:10:43.449838+0000","last_peered":"2024-04-24T18:10:43.449838+0000","last_clean":"2024-04-24T18:10:43.449838+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:43.449838+0000","last_undegraded":"2024-04-24T18:10:43.449838+0000","last_fullsized":"2024-04-24T18:10:43.449838+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":183,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449716+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:10:43.449716+0000","last_peered":"2024-04-24T18:10:43.449716+0000","last_clean":"2024-04-24T18:10:43.449716+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:43.449716+0000","last_undegraded":"2024-04-24T18:10:43.449716+0000","last_fullsized":"2024-04-24T18:10:43.449716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"171'818","reported_seq":6149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329213+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":186,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449934+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:10:43.449934+0000","last_peered":"2024-04-24T18:10:43.449934+0000","last_clean":"2024-04-24T18:10:43.449934+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:43.449934+0000","last_undegraded":"2024-04-24T18:10:43.449934+0000","last_fullsized":"2024-04-24T18:10:43.449934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"165'696","reported_seq":12152,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450131+0000","last_change":"2024-04-24T18:10:20.328757+0000","last_active":"2024-04-24T18:10:43.450131+0000","last_peered":"2024-04-24T18:10:43.450131+0000","last_clean":"2024-04-24T18:10:43.450131+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:10:43.450131+0000","last_undegraded":"2024-04-24T18:10:43.450131+0000","last_fullsized":"2024-04-24T18:10:43.450131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"171'666","reported_seq":6306,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449742+0000","last_change":"2024-04-24T18:10:20.331718+0000","last_active":"2024-04-24T18:10:43.449742+0000","last_peered":"2024-04-24T18:10:43.449742+0000","last_clean":"2024-04-24T18:10:43.449742+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:43.449742+0000","last_undegraded":"2024-04-24T18:10:43.449742+0000","last_fullsized":"2024-04-24T18:10:43.449742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"167'694","reported_seq":3935,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:20.328410+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"171'682","reported_seq":6313,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444827+0000","last_change":"2024-04-24T18:10:21.332171+0000","last_active":"2024-04-24T18:10:43.444827+0000","last_peered":"2024-04-24T18:10:43.444827+0000","last_clean":"2024-04-24T18:10:43.444827+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:43.444827+0000","last_undegraded":"2024-04-24T18:10:43.444827+0000","last_fullsized":"2024-04-24T18:10:43.444827+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:52:41.461799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"171'580","reported_seq":7072,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449978+0000","last_change":"2024-04-24T18:10:20.332029+0000","last_active":"2024-04-24T18:10:43.449978+0000","last_peered":"2024-04-24T18:10:43.449978+0000","last_clean":"2024-04-24T18:10:43.449978+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:10:43.449978+0000","last_undegraded":"2024-04-24T18:10:43.449978+0000","last_fullsized":"2024-04-24T18:10:43.449978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"167'634","reported_seq":9280,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449993+0000","last_change":"2024-04-24T18:10:20.334138+0000","last_active":"2024-04-24T18:10:43.449993+0000","last_peered":"2024-04-24T18:10:43.449993+0000","last_clean":"2024-04-24T18:10:43.449993+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:43.449993+0000","last_undegraded":"2024-04-24T18:10:43.449993+0000","last_fullsized":"2024-04-24T18:10:43.449993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"167'596","reported_seq":7179,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449905+0000","last_change":"2024-04-24T18:10:20.334800+0000","last_active":"2024-04-24T18:10:43.449905+0000","last_peered":"2024-04-24T18:10:43.449905+0000","last_clean":"2024-04-24T18:10:43.449905+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:43.449905+0000","last_undegraded":"2024-04-24T18:10:43.449905+0000","last_fullsized":"2024-04-24T18:10:43.449905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"167'868","reported_seq":6431,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445558+0000","last_change":"2024-04-24T18:10:20.326786+0000","last_active":"2024-04-24T18:10:43.445558+0000","last_peered":"2024-04-24T18:10:43.445558+0000","last_clean":"2024-04-24T18:10:43.445558+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:10:43.445558+0000","last_undegraded":"2024-04-24T18:10:43.445558+0000","last_fullsized":"2024-04-24T18:10:43.445558+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"167'663","reported_seq":7482,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449450+0000","last_change":"2024-04-24T18:10:20.329149+0000","last_active":"2024-04-24T18:10:43.449450+0000","last_peered":"2024-04-24T18:10:43.449450+0000","last_clean":"2024-04-24T18:10:43.449450+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:43.449450+0000","last_undegraded":"2024-04-24T18:10:43.449450+0000","last_fullsized":"2024-04-24T18:10:43.449450+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"171'765","reported_seq":9197,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329631+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"174'662","reported_seq":6522,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444247+0000","last_change":"2024-04-24T18:10:20.325240+0000","last_active":"2024-04-24T18:10:43.444247+0000","last_peered":"2024-04-24T18:10:43.444247+0000","last_clean":"2024-04-24T18:10:43.444247+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:43.444247+0000","last_undegraded":"2024-04-24T18:10:43.444247+0000","last_fullsized":"2024-04-24T18:10:43.444247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"167'967","reported_seq":9149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449919+0000","last_change":"2024-04-24T18:10:20.334115+0000","last_active":"2024-04-24T18:10:43.449919+0000","last_peered":"2024-04-24T18:10:43.449919+0000","last_clean":"2024-04-24T18:10:43.449919+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:10:43.449919+0000","last_undegraded":"2024-04-24T18:10:43.449919+0000","last_fullsized":"2024-04-24T18:10:43.449919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"167'688","reported_seq":8040,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449785+0000","last_change":"2024-04-24T18:10:20.330052+0000","last_active":"2024-04-24T18:10:43.449785+0000","last_peered":"2024-04-24T18:10:43.449785+0000","last_clean":"2024-04-24T18:10:43.449785+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:10:43.449785+0000","last_undegraded":"2024-04-24T18:10:43.449785+0000","last_fullsized":"2024-04-24T18:10:43.449785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"167'621","reported_seq":6044,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450254+0000","last_change":"2024-04-24T18:10:20.329098+0000","last_active":"2024-04-24T18:10:43.450254+0000","last_peered":"2024-04-24T18:10:43.450254+0000","last_clean":"2024-04-24T18:10:43.450254+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:10:43.450254+0000","last_undegraded":"2024-04-24T18:10:43.450254+0000","last_fullsized":"2024-04-24T18:10:43.450254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"171'1163","reported_seq":7009,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445089+0000","last_change":"2024-04-24T18:10:22.337859+0000","last_active":"2024-04-24T18:10:43.445089+0000","last_peered":"2024-04-24T18:10:43.445089+0000","last_clean":"2024-04-24T18:10:43.445089+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:43.445089+0000","last_undegraded":"2024-04-24T18:10:43.445089+0000","last_fullsized":"2024-04-24T18:10:43.445089+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:26:55.796441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":26264,"ondisk_log_size":26264,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574894,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307912,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063928,"statfs":{"total":96636764160,"available":96321462272,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53000000000000003,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48599999999999999,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.67000000000000004,"5min":2.585,"15min":2.585},"last":0.64800000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78300000000000003,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":2.008,"15min":2.008},"last":0.47999999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.55100000000000005,"15min":0.51000000000000001},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.505},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.35499999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.42999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.753,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":1.196,"15min":1.196},"last":0.69199999999999995}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":1.131,"15min":1.131},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77900000000000003,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.622},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.75700000000000001}]}]},{"osd":6,"up_from":95,"seq":408021893145,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309128,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":306784,"kb_avail":94062712,"statfs":{"total":96636764160,"available":96320217088,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":314146847},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":1.0069999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.89100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":10.071999999999999,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.96099999999999997},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.38}]},{"osd":2,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":2.9609999999999999},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.94899999999999995}]},{"osd":3,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.79600000000000004}]},{"osd":4,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.53800000000000003}]},{"osd":5,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.498},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.66600000000000004}]}]},{"osd":1,"up_from":13,"seq":55834574893,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307888,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063952,"statfs":{"total":96636764160,"available":96321486848,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.53500000000000003,"15min":0.61799999999999999},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.5,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.26100000000000001},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.89500000000000002,"15min":1.127},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":14.15,"15min":14.15},"last":0.28199999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.93600000000000005,"5min":1.306,"15min":1.306},"last":0.33300000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.96099999999999997,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.95499999999999996,"5min":14.118,"15min":14.118},"last":0.34699999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.93000000000000005,"15min":1.1519999999999999},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67600000000000005,"5min":14.089,"15min":14.089},"last":0.371},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.313}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":0.61099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71799999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.0600000000000001,"15min":1.274},"min":{"1min":0.35699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.51900000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.76300000000000001},"min":{"1min":0.371,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2310000000000001,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.746,"5min":1.49,"15min":1.49},"last":0.59399999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.073,"15min":1.2929999999999999},"min":{"1min":0.438,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1279999999999999,"5min":14.051,"15min":14.051},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.74399999999999999,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.1870000000000001,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.56299999999999994}]}]},{"osd":0,"up_from":13,"seq":55834574894,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311956,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059884,"statfs":{"total":96636764160,"available":96317321216,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65700000000000003,"5min":1.375,"15min":1.375},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.41299999999999998,"5min":1.2549999999999999,"15min":1.6379999999999999},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90200000000000002,"5min":26.936,"15min":26.936},"last":0.45700000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.92000000000000004,"15min":1.1479999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":11.93,"15min":11.93},"last":0.35699999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42699999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.157,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.49299999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.83599999999999997,"15min":0.995},"min":{"1min":0.25900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.93799999999999994,"5min":14.371,"15min":14.371},"last":0.432},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.86699999999999999,"15min":0.97399999999999998},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.41999999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.92800000000000005,"15min":1.1140000000000001},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":14.414,"15min":14.414},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.93200000000000005,"5min":1.329,"15min":1.329},"last":0.58299999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83099999999999996,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97099999999999997,"15min":1.133},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.995,"5min":14.452,"15min":14.452},"last":0.56499999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.65300000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86099999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.67000000000000004},{"interface":"front","average":{"1min":0.628,"5min":0.98699999999999999,"15min":1.1479999999999999},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1259999999999999,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.60899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574893,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299656,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072184,"statfs":{"total":96636764160,"available":96329916416,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":4.109,"15min":4.109},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.441,"15min":0.42099999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77900000000000003,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.30099999999999999}]},{"osd":1,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.434,"15min":0.40699999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":1.619,"15min":1.619},"last":0.23699999999999999}]},{"osd":3,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.316}]},{"osd":4,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":1.155,"15min":1.155},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.77000000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.751},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.55400000000000005}]},{"osd":7,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.71799999999999997,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":10.516,"15min":10.516},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.56999999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574894,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303764,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2367488,"data_stored":1201291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36299999999999999,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.41799999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.26400000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.28499999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48099999999999998,"5min":1.792,"15min":1.792},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.44800000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.50900000000000001,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.436}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68400000000000005,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.34599999999999997},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.76300000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.40300000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70699999999999996,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.39900000000000002}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.377}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.755,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.499},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.74299999999999999,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":15.4,"15min":15.4},"last":0.53200000000000003}]}]},{"osd":4,"up_from":13,"seq":55834574894,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311952,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059888,"statfs":{"total":96636764160,"available":96317325312,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46800000000000003,"15min":0.47499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.5},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.497},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55000000000000004,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.53700000000000003},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.626,"5min":0.627,"15min":0.627},"last":0.371}]},{"osd":2,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.33900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.27600000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63800000000000001,"5min":1.153,"15min":1.153},"last":0.40500000000000003}]},{"osd":3,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.58399999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":1.373,"15min":1.373},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.59799999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71099999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.60999999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574894,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2363392,"data_stored":1201272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.307,"5min":0.189,"15min":0.189},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":3.5840000000000001},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.39600000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":1.3520000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":3.7130000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":1.296,"15min":1.296},"last":1.407},{"interface":"front","average":{"1min":0.44,"5min":0.504,"15min":0.53100000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63300000000000001,"5min":1.377,"15min":1.377},"last":1.361}]},{"osd":3,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":1.899,"15min":1.899},"last":0.98299999999999998},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59899999999999998,"5min":1.881,"15min":1.881},"last":0.997}]},{"osd":4,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46999999999999997,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.66100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.376},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52000000000000002,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.432}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.46000000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":1.016,"15min":1.016},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54400000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.48999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:10:50.410 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-24T18:10:50.743 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:50.744 INFO:teuthology.orchestra.run.smithi049.stdout:{"epoch":195,"fsid":"5dae32d6-9101-4f7b-a6ef-ca5186fee71b","created":"2024-04-24T18:07:07.515859+0000","modified":"2024-04-24T18:10:44.445859+0000","last_up_change":"2024-04-24T18:08:52.865354+0000","last_in_change":"2024-04-24T18:10:17.313880+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":"rbd","create_time":"2024-04-24T18:07:18.850062+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[3],"tier_of":-1,"read_tier":3,"write_tier":3,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":2,"pool_name":".mgr","create_time":"2024-04-24T18:07:19.353026+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":3,"pool_name":"cache","create_time":"2024-04-24T18:07:25.800822+0000","flags":9,"flags_names":"hashpspool,incomplete_clones","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":4,"pg_placement_num":4,"pg_placement_num_target":4,"pg_num_target":4,"pg_num_pending":4,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"21","last_force_op_resend_prenautilus":"21","last_force_op_resend_preluminous":"21","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":19,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":1,"read_tier":-1,"write_tier":-1,"cache_mode":"writeback","target_max_bytes":0,"target_max_objects":250,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"bloom","false_positive_probability":0.050000000000000003,"target_size":0,"seed":0},"hit_set_period":60,"hit_set_count":8,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[1000000,1000000,1000000,1000000,1000000,1000000,1000000,1000000],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"pool_client.0","create_time":"2024-04-24T18:07:35.513947+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":107,"snap_epoch":172,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"24da2d2a-953e-4440-9f6d-fb19cbec9ca1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":173,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6801","nonce":43474},{"type":"v1","addr":"172.21.15.49:6803","nonce":43474}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6805","nonce":43474},{"type":"v1","addr":"172.21.15.49:6807","nonce":43474}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6813","nonce":43474},{"type":"v1","addr":"172.21.15.49:6815","nonce":43474}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6809","nonce":43474},{"type":"v1","addr":"172.21.15.49:6811","nonce":43474}]},"public_addr":"172.21.15.49:6803/43474","cluster_addr":"172.21.15.49:6807/43474","heartbeat_back_addr":"172.21.15.49:6815/43474","heartbeat_front_addr":"172.21.15.49:6811/43474","state":["exists","up"]},{"osd":1,"uuid":"a47a3673-abbd-4827-966a-d68b17f3a18f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6800","nonce":43473},{"type":"v1","addr":"172.21.15.49:6802","nonce":43473}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6804","nonce":43473},{"type":"v1","addr":"172.21.15.49:6806","nonce":43473}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6812","nonce":43473},{"type":"v1","addr":"172.21.15.49:6814","nonce":43473}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6808","nonce":43473},{"type":"v1","addr":"172.21.15.49:6810","nonce":43473}]},"public_addr":"172.21.15.49:6802/43473","cluster_addr":"172.21.15.49:6806/43473","heartbeat_back_addr":"172.21.15.49:6814/43473","heartbeat_front_addr":"172.21.15.49:6810/43473","state":["exists","up"]},{"osd":2,"uuid":"de3910ae-cc81-45d4-8639-cf2415330e64","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":177,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6824","nonce":43481},{"type":"v1","addr":"172.21.15.49:6825","nonce":43481}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6826","nonce":43481},{"type":"v1","addr":"172.21.15.49:6827","nonce":43481}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6830","nonce":43481},{"type":"v1","addr":"172.21.15.49:6831","nonce":43481}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6828","nonce":43481},{"type":"v1","addr":"172.21.15.49:6829","nonce":43481}]},"public_addr":"172.21.15.49:6825/43481","cluster_addr":"172.21.15.49:6827/43481","heartbeat_back_addr":"172.21.15.49:6831/43481","heartbeat_front_addr":"172.21.15.49:6829/43481","state":["exists","up"]},{"osd":3,"uuid":"20f044c6-a9b5-4eaa-b25c-0de3ef8dffb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":172,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6816","nonce":43475},{"type":"v1","addr":"172.21.15.49:6817","nonce":43475}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6818","nonce":43475},{"type":"v1","addr":"172.21.15.49:6819","nonce":43475}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6822","nonce":43475},{"type":"v1","addr":"172.21.15.49:6823","nonce":43475}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.49:6820","nonce":43475},{"type":"v1","addr":"172.21.15.49:6821","nonce":43475}]},"public_addr":"172.21.15.49:6817/43475","cluster_addr":"172.21.15.49:6819/43475","heartbeat_back_addr":"172.21.15.49:6823/43475","heartbeat_front_addr":"172.21.15.49:6821/43475","state":["exists","up"]},{"osd":4,"uuid":"47f1fd84-e601-4e48-8058-bfa9296a9f90","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6800","nonce":43499},{"type":"v1","addr":"172.21.15.78:6801","nonce":43499}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6802","nonce":43499},{"type":"v1","addr":"172.21.15.78:6803","nonce":43499}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6806","nonce":43499},{"type":"v1","addr":"172.21.15.78:6807","nonce":43499}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6804","nonce":43499},{"type":"v1","addr":"172.21.15.78:6805","nonce":43499}]},"public_addr":"172.21.15.78:6801/43499","cluster_addr":"172.21.15.78:6803/43499","heartbeat_back_addr":"172.21.15.78:6807/43499","heartbeat_front_addr":"172.21.15.78:6805/43499","state":["exists","up"]},{"osd":5,"uuid":"1796d83b-591c-4c9d-b376-e36a2416a17e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6808","nonce":43500},{"type":"v1","addr":"172.21.15.78:6809","nonce":43500}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6810","nonce":43500},{"type":"v1","addr":"172.21.15.78:6811","nonce":43500}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6814","nonce":43500},{"type":"v1","addr":"172.21.15.78:6815","nonce":43500}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6812","nonce":43500},{"type":"v1","addr":"172.21.15.78:6813","nonce":43500}]},"public_addr":"172.21.15.78:6809/43500","cluster_addr":"172.21.15.78:6811/43500","heartbeat_back_addr":"172.21.15.78:6815/43500","heartbeat_front_addr":"172.21.15.78:6813/43500","state":["exists","up"]},{"osd":6,"uuid":"5f203bac-3db4-4129-96cc-63dc06ee5ffa","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":67,"up_from":95,"up_thru":178,"down_at":69,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6824","nonce":58122},{"type":"v1","addr":"172.21.15.78:6825","nonce":58122}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6826","nonce":58122},{"type":"v1","addr":"172.21.15.78:6827","nonce":58122}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6830","nonce":58122},{"type":"v1","addr":"172.21.15.78:6831","nonce":58122}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6828","nonce":58122},{"type":"v1","addr":"172.21.15.78:6829","nonce":58122}]},"public_addr":"172.21.15.78:6825/58122","cluster_addr":"172.21.15.78:6827/58122","heartbeat_back_addr":"172.21.15.78:6831/58122","heartbeat_front_addr":"172.21.15.78:6829/58122","state":["exists","up"]},{"osd":7,"uuid":"51061805-fbe5-4e80-be6b-dfc37cec0221","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":169,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6816","nonce":43501},{"type":"v1","addr":"172.21.15.78:6817","nonce":43501}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6818","nonce":43501},{"type":"v1","addr":"172.21.15.78:6819","nonce":43501}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6822","nonce":43501},{"type":"v1","addr":"172.21.15.78:6823","nonce":43501}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.78:6820","nonce":43501},{"type":"v1","addr":"172.21.15.78:6821","nonce":43501}]},"public_addr":"172.21.15.78:6817/43501","cluster_addr":"172.21.15.78:6819/43501","heartbeat_back_addr":"172.21.15.78:6823/43501","heartbeat_front_addr":"172.21.15.78:6821/43501","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.248445+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.246633+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.176542+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.192111+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.151845+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.187089+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.255937+0000","dead_epoch":69},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-04-24T18:07:15.160381+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-24T18:10:51.757 INFO:tasks.ceph:Scrubbing osd.0 2024-04-24T18:10:51.758 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:51.919 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:51.920 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:51.920 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:51.932 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-24T18:10:52.265 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 0 to deep-scrub 2024-04-24T18:10:52.276 INFO:tasks.ceph:Scrubbing osd.1 2024-04-24T18:10:52.277 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:52.424 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:52.424 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:52.424 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:52.436 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-24T18:10:52.773 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 1 to deep-scrub 2024-04-24T18:10:52.783 INFO:tasks.ceph:Scrubbing osd.2 2024-04-24T18:10:52.784 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:52.930 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:52.930 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:52.930 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:52.942 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-24T18:10:53.276 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 2 to deep-scrub 2024-04-24T18:10:53.287 INFO:tasks.ceph:Scrubbing osd.3 2024-04-24T18:10:53.288 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:53.435 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:53.435 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:53.435 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:53.447 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-24T18:10:53.782 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 3 to deep-scrub 2024-04-24T18:10:53.794 INFO:tasks.ceph:Scrubbing osd.4 2024-04-24T18:10:53.794 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:53.942 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:53.942 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:53.942 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:53.954 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-04-24T18:10:54.298 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 4 to deep-scrub 2024-04-24T18:10:54.309 INFO:tasks.ceph:Scrubbing osd.5 2024-04-24T18:10:54.309 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:54.456 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:54.457 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:54.457 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:54.469 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-04-24T18:10:54.800 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 5 to deep-scrub 2024-04-24T18:10:54.811 INFO:tasks.ceph:Scrubbing osd.6 2024-04-24T18:10:54.811 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:54.958 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:54.958 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:54.958 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:54.970 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-04-24T18:10:55.315 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 6 to deep-scrub 2024-04-24T18:10:55.326 INFO:tasks.ceph:Scrubbing osd.7 2024-04-24T18:10:55.327 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-04-24T18:10:55.474 INFO:teuthology.orchestra.run.smithi049.stdout:{ 2024-04-24T18:10:55.475 INFO:teuthology.orchestra.run.smithi049.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-24T18:10:55.475 INFO:teuthology.orchestra.run.smithi049.stdout:} 2024-04-24T18:10:55.487 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-04-24T18:10:55.822 INFO:teuthology.orchestra.run.smithi049.stderr:instructed osd(s) 7 to deep-scrub 2024-04-24T18:10:55.834 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:10:56.188 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:10:56.189 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:10:56.201 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":297,"stamp":"2024-04-24T18:10:55.339201+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590640,"num_objects":1800,"num_object_clones":0,"num_object_copies":3600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29264,"num_write_kb":534157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":26387,"ondisk_log_size":26387,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2456016,"kb_used_data":18528,"kb_used_omap":43,"kb_used_meta":2437332,"kb_avail":752518704,"statfs":{"total":773094113280,"available":770579152896,"internally_reserved":0,"allocated":18972672,"data_stored":10790096,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2495828359},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.890254"},"pg_stats":[{"pgid":"4.1f","version":"173'683","reported_seq":7230,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449490+0000","last_change":"2024-04-24T18:10:27.328087+0000","last_active":"2024-04-24T18:10:43.449490+0000","last_peered":"2024-04-24T18:10:43.449490+0000","last_clean":"2024-04-24T18:10:43.449490+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:10:43.449490+0000","last_undegraded":"2024-04-24T18:10:43.449490+0000","last_fullsized":"2024-04-24T18:10:43.449490+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":683,"log_dups_size":0,"ondisk_log_size":683,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:33:17.307412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"160'1339","reported_seq":7258,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:44.450574+0000","last_change":"2024-04-24T18:10:21.335419+0000","last_active":"2024-04-24T18:10:44.450574+0000","last_peered":"2024-04-24T18:10:44.450574+0000","last_clean":"2024-04-24T18:10:44.450574+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:44.450574+0000","last_undegraded":"2024-04-24T18:10:44.450574+0000","last_fullsized":"2024-04-24T18:10:44.450574+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1339,"log_dups_size":0,"ondisk_log_size":1339,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"167'710","reported_seq":5997,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449714+0000","last_change":"2024-04-24T18:10:20.330289+0000","last_active":"2024-04-24T18:10:43.449714+0000","last_peered":"2024-04-24T18:10:43.449714+0000","last_clean":"2024-04-24T18:10:43.449714+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:10:43.449714+0000","last_undegraded":"2024-04-24T18:10:43.449714+0000","last_fullsized":"2024-04-24T18:10:43.449714+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":710,"log_dups_size":0,"ondisk_log_size":710,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:15:57.552319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"173'1079","reported_seq":10264,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445678+0000","last_change":"2024-04-24T18:10:20.345289+0000","last_active":"2024-04-24T18:10:43.445678+0000","last_peered":"2024-04-24T18:10:43.445678+0000","last_clean":"2024-04-24T18:10:43.445678+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:43.445678+0000","last_undegraded":"2024-04-24T18:10:43.445678+0000","last_fullsized":"2024-04-24T18:10:43.445678+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1079,"log_dups_size":0,"ondisk_log_size":1079,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:22:26.759931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"171'535","reported_seq":11656,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449807+0000","last_change":"2024-04-24T18:10:20.332101+0000","last_active":"2024-04-24T18:10:43.449807+0000","last_peered":"2024-04-24T18:10:43.449807+0000","last_clean":"2024-04-24T18:10:43.449807+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:43.449807+0000","last_undegraded":"2024-04-24T18:10:43.449807+0000","last_fullsized":"2024-04-24T18:10:43.449807+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:47:47.077821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"167'880","reported_seq":12337,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449756+0000","last_change":"2024-04-24T18:10:20.331679+0000","last_active":"2024-04-24T18:10:43.449756+0000","last_peered":"2024-04-24T18:10:43.449756+0000","last_clean":"2024-04-24T18:10:43.449756+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:43.449756+0000","last_undegraded":"2024-04-24T18:10:43.449756+0000","last_fullsized":"2024-04-24T18:10:43.449756+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":880,"log_dups_size":0,"ondisk_log_size":880,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:51:23.333895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"167'726","reported_seq":9735,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449505+0000","last_change":"2024-04-24T18:10:20.329954+0000","last_active":"2024-04-24T18:10:43.449505+0000","last_peered":"2024-04-24T18:10:43.449505+0000","last_clean":"2024-04-24T18:10:43.449505+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:43.449505+0000","last_undegraded":"2024-04-24T18:10:43.449505+0000","last_fullsized":"2024-04-24T18:10:43.449505+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:04:13.618772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"167'616","reported_seq":11663,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449727+0000","last_change":"2024-04-24T18:10:20.334440+0000","last_active":"2024-04-24T18:10:43.449727+0000","last_peered":"2024-04-24T18:10:43.449727+0000","last_clean":"2024-04-24T18:10:43.449727+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:10:43.449727+0000","last_undegraded":"2024-04-24T18:10:43.449727+0000","last_fullsized":"2024-04-24T18:10:43.449727+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":616,"log_dups_size":0,"ondisk_log_size":616,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:03:10.357762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"167'1362","reported_seq":6969,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449833+0000","last_change":"2024-04-24T18:10:20.329528+0000","last_active":"2024-04-24T18:10:43.449833+0000","last_peered":"2024-04-24T18:10:43.449833+0000","last_clean":"2024-04-24T18:10:43.449833+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:10:43.449833+0000","last_undegraded":"2024-04-24T18:10:43.449833+0000","last_fullsized":"2024-04-24T18:10:43.449833+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1362,"log_dups_size":0,"ondisk_log_size":1362,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:46:56.177431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"174'1237","reported_seq":4971,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444127+0000","last_change":"2024-04-24T18:10:20.324905+0000","last_active":"2024-04-24T18:10:43.444127+0000","last_peered":"2024-04-24T18:10:43.444127+0000","last_clean":"2024-04-24T18:10:43.444127+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:10:43.444127+0000","last_undegraded":"2024-04-24T18:10:43.444127+0000","last_fullsized":"2024-04-24T18:10:43.444127+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1237,"log_dups_size":0,"ondisk_log_size":1237,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:59:23.805555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":159,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449538+0000","last_change":"2024-04-24T18:08:53.874638+0000","last_active":"2024-04-24T18:10:43.449538+0000","last_peered":"2024-04-24T18:10:43.449538+0000","last_clean":"2024-04-24T18:10:43.449538+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:10:43.449538+0000","last_undegraded":"2024-04-24T18:10:43.449538+0000","last_fullsized":"2024-04-24T18:10:43.449538+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:56.777486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":177,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:18.331936+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T01:48:31.195614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"174'957","reported_seq":7181,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:44.450309+0000","last_change":"2024-04-24T18:10:21.336769+0000","last_active":"2024-04-24T18:10:44.450309+0000","last_peered":"2024-04-24T18:10:44.450309+0000","last_clean":"2024-04-24T18:10:44.450309+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:44.450309+0000","last_undegraded":"2024-04-24T18:10:44.450309+0000","last_fullsized":"2024-04-24T18:10:44.450309+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":957,"log_dups_size":0,"ondisk_log_size":957,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449814+0000","last_change":"2024-04-24T18:10:18.330231+0000","last_active":"2024-04-24T18:10:43.449814+0000","last_peered":"2024-04-24T18:10:43.449814+0000","last_clean":"2024-04-24T18:10:43.449814+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:43.449814+0000","last_undegraded":"2024-04-24T18:10:43.449814+0000","last_fullsized":"2024-04-24T18:10:43.449814+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:00:41.033763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":204,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450179+0000","last_change":"2024-04-24T18:10:18.328460+0000","last_active":"2024-04-24T18:10:43.450179+0000","last_peered":"2024-04-24T18:10:43.450179+0000","last_clean":"2024-04-24T18:10:43.450179+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:43.450179+0000","last_undegraded":"2024-04-24T18:10:43.450179+0000","last_fullsized":"2024-04-24T18:10:43.450179+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:04.838043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"165'1061","reported_seq":6785,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444268+0000","last_change":"2024-04-24T18:10:20.326388+0000","last_active":"2024-04-24T18:10:43.444268+0000","last_peered":"2024-04-24T18:10:43.444268+0000","last_clean":"2024-04-24T18:10:43.444268+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:43.444268+0000","last_undegraded":"2024-04-24T18:10:43.444268+0000","last_fullsized":"2024-04-24T18:10:43.444268+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1061,"log_dups_size":0,"ondisk_log_size":1061,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:30:33.503729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"159'4","reported_seq":184,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449661+0000","last_change":"2024-04-24T18:07:27.505051+0000","last_active":"2024-04-24T18:10:43.449661+0000","last_peered":"2024-04-24T18:10:43.449661+0000","last_clean":"2024-04-24T18:10:43.449661+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:43.449661+0000","last_undegraded":"2024-04-24T18:10:43.449661+0000","last_fullsized":"2024-04-24T18:10:43.449661+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:36:03.892758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450353+0000","last_change":"2024-04-24T18:10:18.331833+0000","last_active":"2024-04-24T18:10:43.450353+0000","last_peered":"2024-04-24T18:10:43.450353+0000","last_clean":"2024-04-24T18:10:43.450353+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:43.450353+0000","last_undegraded":"2024-04-24T18:10:43.450353+0000","last_fullsized":"2024-04-24T18:10:43.450353+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:03:47.993822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"168'981","reported_seq":10279,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449459+0000","last_change":"2024-04-24T18:10:20.327837+0000","last_active":"2024-04-24T18:10:43.449459+0000","last_peered":"2024-04-24T18:10:43.449459+0000","last_clean":"2024-04-24T18:10:43.449459+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:10:43.449459+0000","last_undegraded":"2024-04-24T18:10:43.449459+0000","last_fullsized":"2024-04-24T18:10:43.449459+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":981,"log_dups_size":0,"ondisk_log_size":981,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:15:40.995869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":158,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444134+0000","last_change":"2024-04-24T18:07:56.633007+0000","last_active":"2024-04-24T18:10:43.444134+0000","last_peered":"2024-04-24T18:10:43.444134+0000","last_clean":"2024-04-24T18:10:43.444134+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:10:43.444134+0000","last_undegraded":"2024-04-24T18:10:43.444134+0000","last_fullsized":"2024-04-24T18:10:43.444134+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:04.382312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":201,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450351+0000","last_change":"2024-04-24T18:10:18.331400+0000","last_active":"2024-04-24T18:10:43.450351+0000","last_peered":"2024-04-24T18:10:43.450351+0000","last_clean":"2024-04-24T18:10:43.450351+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:10:43.450351+0000","last_undegraded":"2024-04-24T18:10:43.450351+0000","last_fullsized":"2024-04-24T18:10:43.450351+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:01:18.213539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"167'574","reported_seq":5342,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444387+0000","last_change":"2024-04-24T18:10:20.327045+0000","last_active":"2024-04-24T18:10:43.444387+0000","last_peered":"2024-04-24T18:10:43.444387+0000","last_clean":"2024-04-24T18:10:43.444387+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:43.444387+0000","last_undegraded":"2024-04-24T18:10:43.444387+0000","last_fullsized":"2024-04-24T18:10:43.444387+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":574,"log_dups_size":0,"ondisk_log_size":574,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:18:26.759394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"113'2","reported_seq":41,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:44.450159+0000","last_change":"2024-04-24T18:10:18.333879+0000","last_active":"2024-04-24T18:10:44.450159+0000","last_peered":"2024-04-24T18:10:44.450159+0000","last_clean":"2024-04-24T18:10:44.450159+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:44.450159+0000","last_undegraded":"2024-04-24T18:10:44.450159+0000","last_fullsized":"2024-04-24T18:10:44.450159+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:26.488219+0000","last_clean_scrub_stamp":"2024-04-24T18:07:26.488219+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"17'117","reported_seq":381,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450453+0000","last_change":"2024-04-24T18:07:59.632798+0000","last_active":"2024-04-24T18:10:43.450453+0000","last_peered":"2024-04-24T18:10:43.450453+0000","last_clean":"2024-04-24T18:10:43.450453+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:10:43.450453+0000","last_undegraded":"2024-04-24T18:10:43.450453+0000","last_fullsized":"2024-04-24T18:10:43.450453+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:42.276057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":173,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444276+0000","last_change":"2024-04-24T18:08:53.875419+0000","last_active":"2024-04-24T18:10:43.444276+0000","last_peered":"2024-04-24T18:10:43.444276+0000","last_clean":"2024-04-24T18:10:43.444276+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:10:43.444276+0000","last_undegraded":"2024-04-24T18:10:43.444276+0000","last_fullsized":"2024-04-24T18:10:43.444276+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:09:28.877323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"174'1090","reported_seq":10186,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:44.450766+0000","last_change":"2024-04-24T18:10:22.338092+0000","last_active":"2024-04-24T18:10:44.450766+0000","last_peered":"2024-04-24T18:10:44.450766+0000","last_clean":"2024-04-24T18:10:44.450766+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:10:44.450766+0000","last_undegraded":"2024-04-24T18:10:44.450766+0000","last_fullsized":"2024-04-24T18:10:44.450766+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1090,"log_dups_size":0,"ondisk_log_size":1090,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"174'671","reported_seq":6166,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449838+0000","last_change":"2024-04-24T18:10:20.332296+0000","last_active":"2024-04-24T18:10:43.449838+0000","last_peered":"2024-04-24T18:10:43.449838+0000","last_clean":"2024-04-24T18:10:43.449838+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:43.449838+0000","last_undegraded":"2024-04-24T18:10:43.449838+0000","last_fullsized":"2024-04-24T18:10:43.449838+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:55:01.484938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":183,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449716+0000","last_change":"2024-04-24T18:07:20.405082+0000","last_active":"2024-04-24T18:10:43.449716+0000","last_peered":"2024-04-24T18:10:43.449716+0000","last_clean":"2024-04-24T18:10:43.449716+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:43.449716+0000","last_undegraded":"2024-04-24T18:10:43.449716+0000","last_fullsized":"2024-04-24T18:10:43.449716+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:19:49.691570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"171'818","reported_seq":6149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329213+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":818,"log_dups_size":0,"ondisk_log_size":818,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:34:37.940908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":186,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449934+0000","last_change":"2024-04-24T18:07:21.109584+0000","last_active":"2024-04-24T18:10:43.449934+0000","last_peered":"2024-04-24T18:10:43.449934+0000","last_clean":"2024-04-24T18:10:43.449934+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:43.449934+0000","last_undegraded":"2024-04-24T18:10:43.449934+0000","last_fullsized":"2024-04-24T18:10:43.449934+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:19.379531+0000","last_clean_scrub_stamp":"2024-04-24T18:07:19.379531+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:06:23.536693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"165'696","reported_seq":12152,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450131+0000","last_change":"2024-04-24T18:10:20.328757+0000","last_active":"2024-04-24T18:10:43.450131+0000","last_peered":"2024-04-24T18:10:43.450131+0000","last_clean":"2024-04-24T18:10:43.450131+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:10:43.450131+0000","last_undegraded":"2024-04-24T18:10:43.450131+0000","last_fullsized":"2024-04-24T18:10:43.450131+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":696,"log_dups_size":0,"ondisk_log_size":696,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:15:47.239740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"171'666","reported_seq":6306,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449742+0000","last_change":"2024-04-24T18:10:20.331718+0000","last_active":"2024-04-24T18:10:43.449742+0000","last_peered":"2024-04-24T18:10:43.449742+0000","last_clean":"2024-04-24T18:10:43.449742+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:43.449742+0000","last_undegraded":"2024-04-24T18:10:43.449742+0000","last_fullsized":"2024-04-24T18:10:43.449742+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:59:47.494012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"167'694","reported_seq":3935,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449438+0000","last_change":"2024-04-24T18:10:20.328410+0000","last_active":"2024-04-24T18:10:43.449438+0000","last_peered":"2024-04-24T18:10:43.449438+0000","last_clean":"2024-04-24T18:10:43.449438+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:43.449438+0000","last_undegraded":"2024-04-24T18:10:43.449438+0000","last_fullsized":"2024-04-24T18:10:43.449438+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":694,"log_dups_size":0,"ondisk_log_size":694,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:01:35.924876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"171'682","reported_seq":6314,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:44.450404+0000","last_change":"2024-04-24T18:10:21.332171+0000","last_active":"2024-04-24T18:10:44.450404+0000","last_peered":"2024-04-24T18:10:44.450404+0000","last_clean":"2024-04-24T18:10:44.450404+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:44.450404+0000","last_undegraded":"2024-04-24T18:10:44.450404+0000","last_fullsized":"2024-04-24T18:10:44.450404+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":682,"log_dups_size":0,"ondisk_log_size":682,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"171'580","reported_seq":7072,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449978+0000","last_change":"2024-04-24T18:10:20.332029+0000","last_active":"2024-04-24T18:10:43.449978+0000","last_peered":"2024-04-24T18:10:43.449978+0000","last_clean":"2024-04-24T18:10:43.449978+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:10:43.449978+0000","last_undegraded":"2024-04-24T18:10:43.449978+0000","last_fullsized":"2024-04-24T18:10:43.449978+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":580,"log_dups_size":0,"ondisk_log_size":580,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:34:00.537203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"167'634","reported_seq":9280,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449993+0000","last_change":"2024-04-24T18:10:20.334138+0000","last_active":"2024-04-24T18:10:43.449993+0000","last_peered":"2024-04-24T18:10:43.449993+0000","last_clean":"2024-04-24T18:10:43.449993+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:43.449993+0000","last_undegraded":"2024-04-24T18:10:43.449993+0000","last_fullsized":"2024-04-24T18:10:43.449993+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:37.491442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"167'596","reported_seq":7179,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449905+0000","last_change":"2024-04-24T18:10:20.334800+0000","last_active":"2024-04-24T18:10:43.449905+0000","last_peered":"2024-04-24T18:10:43.449905+0000","last_clean":"2024-04-24T18:10:43.449905+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:43.449905+0000","last_undegraded":"2024-04-24T18:10:43.449905+0000","last_fullsized":"2024-04-24T18:10:43.449905+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:32.910736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"167'868","reported_seq":6431,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.445558+0000","last_change":"2024-04-24T18:10:20.326786+0000","last_active":"2024-04-24T18:10:43.445558+0000","last_peered":"2024-04-24T18:10:43.445558+0000","last_clean":"2024-04-24T18:10:43.445558+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:10:43.445558+0000","last_undegraded":"2024-04-24T18:10:43.445558+0000","last_fullsized":"2024-04-24T18:10:43.445558+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":868,"log_dups_size":0,"ondisk_log_size":868,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:06:39.144957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"167'663","reported_seq":7482,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449450+0000","last_change":"2024-04-24T18:10:20.329149+0000","last_active":"2024-04-24T18:10:43.449450+0000","last_peered":"2024-04-24T18:10:43.449450+0000","last_clean":"2024-04-24T18:10:43.449450+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:43.449450+0000","last_undegraded":"2024-04-24T18:10:43.449450+0000","last_fullsized":"2024-04-24T18:10:43.449450+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:56.937136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"171'765","reported_seq":9197,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449452+0000","last_change":"2024-04-24T18:10:20.329631+0000","last_active":"2024-04-24T18:10:43.449452+0000","last_peered":"2024-04-24T18:10:43.449452+0000","last_clean":"2024-04-24T18:10:43.449452+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:10:43.449452+0000","last_undegraded":"2024-04-24T18:10:43.449452+0000","last_fullsized":"2024-04-24T18:10:43.449452+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":765,"log_dups_size":0,"ondisk_log_size":765,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:07:10.502007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"174'662","reported_seq":6522,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.444247+0000","last_change":"2024-04-24T18:10:20.325240+0000","last_active":"2024-04-24T18:10:43.444247+0000","last_peered":"2024-04-24T18:10:43.444247+0000","last_clean":"2024-04-24T18:10:43.444247+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:43.444247+0000","last_undegraded":"2024-04-24T18:10:43.444247+0000","last_fullsized":"2024-04-24T18:10:43.444247+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":662,"log_dups_size":0,"ondisk_log_size":662,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:14:34.019705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"167'967","reported_seq":9149,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449919+0000","last_change":"2024-04-24T18:10:20.334115+0000","last_active":"2024-04-24T18:10:43.449919+0000","last_peered":"2024-04-24T18:10:43.449919+0000","last_clean":"2024-04-24T18:10:43.449919+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:10:43.449919+0000","last_undegraded":"2024-04-24T18:10:43.449919+0000","last_fullsized":"2024-04-24T18:10:43.449919+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":967,"log_dups_size":0,"ondisk_log_size":967,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:30:11.373273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"167'688","reported_seq":8040,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.449785+0000","last_change":"2024-04-24T18:10:20.330052+0000","last_active":"2024-04-24T18:10:43.449785+0000","last_peered":"2024-04-24T18:10:43.449785+0000","last_clean":"2024-04-24T18:10:43.449785+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:10:43.449785+0000","last_undegraded":"2024-04-24T18:10:43.449785+0000","last_fullsized":"2024-04-24T18:10:43.449785+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":688,"log_dups_size":0,"ondisk_log_size":688,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:12.596920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"167'621","reported_seq":6044,"reported_epoch":194,"state":"active+clean","last_fresh":"2024-04-24T18:10:43.450254+0000","last_change":"2024-04-24T18:10:20.329098+0000","last_active":"2024-04-24T18:10:43.450254+0000","last_peered":"2024-04-24T18:10:43.450254+0000","last_clean":"2024-04-24T18:10:43.450254+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:10:43.450254+0000","last_undegraded":"2024-04-24T18:10:43.450254+0000","last_fullsized":"2024-04-24T18:10:43.450254+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":621,"log_dups_size":0,"ondisk_log_size":621,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:19:58.425704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"171'1163","reported_seq":7010,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:44.450679+0000","last_change":"2024-04-24T18:10:22.337859+0000","last_active":"2024-04-24T18:10:44.450679+0000","last_peered":"2024-04-24T18:10:44.450679+0000","last_clean":"2024-04-24T18:10:44.450679+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:44.450679+0000","last_undegraded":"2024-04-24T18:10:44.450679+0000","last_fullsized":"2024-04-24T18:10:44.450679+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":1163,"log_dups_size":0,"ondisk_log_size":1163,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":26264,"ondisk_log_size":26264,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":6,"ondisk_log_size":6,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574895,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307912,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063928,"statfs":{"total":96636764160,"available":96321462272,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53000000000000003,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.376},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48599999999999999,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.67000000000000004,"5min":2.585,"15min":2.585},"last":0.39400000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78300000000000003,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":2.008,"15min":2.008},"last":0.34899999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.55100000000000005,"15min":0.51000000000000001},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.47899999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.42999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.753,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":1.196,"15min":1.196},"last":0.65300000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":1.131,"15min":1.131},"last":0.56499999999999995}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77900000000000003,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.53700000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.51800000000000002},"min":{"1min":0.34499999999999997,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":1.1579999999999999,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.67600000000000005}]}]},{"osd":6,"up_from":95,"seq":408021893146,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":309128,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":306784,"kb_avail":94062712,"statfs":{"total":96636764160,"available":96320217088,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":314146847},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.434,"15min":0.434},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.3260000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.501}]},{"osd":1,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.95199999999999996,"5min":0.95199999999999996,"15min":0.95199999999999996},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":10.071999999999999,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":1.26,"5min":1.26,"15min":1.26},"last":0.48199999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56299999999999994,"15min":0.56299999999999994},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.3940000000000001,"5min":2.3940000000000001,"15min":2.3940000000000001},"last":0.442},{"interface":"front","average":{"1min":0.621,"5min":0.621,"15min":0.621},"min":{"1min":0.314,"5min":0.314,"15min":0.314},"max":{"1min":2.306,"5min":2.306,"15min":2.306},"last":0.54000000000000004}]},{"osd":3,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.61899999999999999,"15min":0.61899999999999999},"min":{"1min":0.34399999999999997,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":2.468,"5min":2.468,"15min":2.468},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.4300000000000002,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65800000000000003,"5min":0.65800000000000003,"15min":0.65800000000000003},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.59899999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.34699999999999998,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":2.3599999999999999,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.58399999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:09:55 2024","interfaces":[{"interface":"back","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":2.2669999999999999,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.68500000000000005,"5min":0.68500000000000005,"15min":0.68500000000000005},"last":0.61099999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574894,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307888,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063952,"statfs":{"total":96636764160,"available":96321486848,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.53500000000000003,"15min":0.61799999999999999},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.5,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.89500000000000002,"15min":1.127},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":14.15,"15min":14.15},"last":0.33100000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.93600000000000005,"5min":1.306,"15min":1.306},"last":0.27100000000000002},{"interface":"front","average":{"1min":0.433,"5min":0.96099999999999997,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.95499999999999996,"5min":14.118,"15min":14.118},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.93000000000000005,"15min":1.1519999999999999},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67600000000000005,"5min":14.089,"15min":14.089},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.36799999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":0.61099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71799999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.0600000000000001,"15min":1.274},"min":{"1min":0.35699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.48499999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.76300000000000001},"min":{"1min":0.371,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2310000000000001,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.498},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.746,"5min":1.49,"15min":1.49},"last":0.40500000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.70799999999999996},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.57099999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.073,"15min":1.2929999999999999},"min":{"1min":0.438,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1279999999999999,"5min":14.051,"15min":14.051},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.74399999999999999,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.1870000000000001,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.434}]}]},{"osd":0,"up_from":13,"seq":55834574895,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311956,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059884,"statfs":{"total":96636764160,"available":96317321216,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65700000000000003,"5min":1.375,"15min":1.375},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.41299999999999998,"5min":1.2549999999999999,"15min":1.6379999999999999},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90200000000000002,"5min":26.936,"15min":26.936},"last":0.435}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.92000000000000004,"15min":1.1479999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":11.93,"15min":11.93},"last":0.30099999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42699999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.157,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.35599999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.83599999999999997,"15min":0.995},"min":{"1min":0.25900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.93799999999999994,"5min":14.371,"15min":14.371},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.86699999999999999,"15min":0.97399999999999998},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.317}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.92800000000000005,"15min":1.1140000000000001},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":14.414,"15min":14.414},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.93200000000000005,"5min":1.329,"15min":1.329},"last":0.46899999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83099999999999996,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97099999999999997,"15min":1.133},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.995,"5min":14.452,"15min":14.452},"last":0.54500000000000004}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.58899999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.86599999999999999,"5min":0.86599999999999999,"15min":0.86599999999999999},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86099999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.628,"5min":0.98699999999999999,"15min":1.1479999999999999},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1259999999999999,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.64500000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574894,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":299656,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":94072184,"statfs":{"total":96636764160,"available":96329916416,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":4.109,"15min":4.109},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.441,"15min":0.42099999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77900000000000003,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.39600000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.434,"15min":0.40699999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.24099999999999999},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":1.619,"15min":1.619},"last":0.318}]},{"osd":3,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.308},{"interface":"front","average":{"1min":0.439,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.377}]},{"osd":4,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":1.155,"15min":1.155},"last":0.70299999999999996}]},{"osd":5,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.68500000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:09:56 2024","interfaces":[{"interface":"back","average":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":10.587999999999999,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.59399999999999997,"15min":0.59399999999999997},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.66100000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.71799999999999997,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":10.516,"15min":10.516},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.58099999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574895,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303764,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068076,"statfs":{"total":96636764160,"available":96325709824,"internally_reserved":0,"allocated":2367488,"data_stored":1201291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36299999999999999,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.41799999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.34499999999999997},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.23100000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48099999999999998,"5min":1.792,"15min":1.792},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.28100000000000003}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.29399999999999998},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.44800000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.50900000000000001,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.371}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68400000000000005,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.76300000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.49099999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.503},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70699999999999996,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.42499999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.7389999999999999,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":2.7669999999999999,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.54100000000000004}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.755,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.74299999999999999,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":15.4,"15min":15.4},"last":0.56100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574895,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311952,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059888,"statfs":{"total":96636764160,"available":96317325312,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46800000000000003,"15min":0.47499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.497},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.67300000000000004}]},{"osd":1,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55000000000000004,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.626,"5min":0.627,"15min":0.627},"last":0.498}]},{"osd":2,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.33900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.27600000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63800000000000001,"5min":1.153,"15min":1.153},"last":0.69899999999999995}]},{"osd":3,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.57199999999999995}]},{"osd":5,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":1.373,"15min":1.373},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.39100000000000001}]},{"osd":6,"last update":"Wed Apr 24 18:09:54 2024","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.40200000000000002,"15min":0.40200000000000002},"min":{"1min":0.248,"5min":0.248,"15min":0.248},"max":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"last":0.374},{"interface":"front","average":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"last":0.42599999999999999}]},{"osd":7,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71099999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.44},{"interface":"front","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.41299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574895,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303760,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068080,"statfs":{"total":96636764160,"available":96325713920,"internally_reserved":0,"allocated":2363392,"data_stored":1201272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.307,"5min":0.189,"15min":0.189},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66800000000000004}]},{"osd":1,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.5}]},{"osd":2,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":1.296,"15min":1.296},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.504,"15min":0.53100000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63300000000000001,"5min":1.377,"15min":1.377},"last":0.58999999999999997}]},{"osd":3,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":1.899,"15min":1.899},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59899999999999998,"5min":1.881,"15min":1.881},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46999999999999997,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.66100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52000000000000002,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.46700000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:09:53 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.623,"5min":0.623,"15min":0.623},"last":0.54500000000000004}]},{"osd":7,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":1.016,"15min":1.016},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54400000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.63600000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.207 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-24T18:07:26.488219+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=26, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-24T18:07:26.488219+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=26, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.208 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-24T18:07:26.488219+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=26, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-24T18:07:26.488219+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=26, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.209 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2024-04-24T18:07:19.379531+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=19, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.210 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:10:56.211 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-24T18:11:16.212 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:11:16.574 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:11:16.575 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:11:16.586 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":307,"stamp":"2024-04-24T18:11:15.343908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590640,"num_objects":1800,"num_object_clones":0,"num_object_copies":3600,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29264,"num_write_kb":534157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":27784,"ondisk_log_size":27784,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2472352,"kb_used_data":18528,"kb_used_omap":43,"kb_used_meta":2453716,"kb_avail":752502368,"statfs":{"total":773094113280,"available":770562424832,"internally_reserved":0,"allocated":18972672,"data_stored":10790096,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2512605575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002915"},"pg_stats":[{"pgid":"4.1f","version":"195'756","reported_seq":7316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:09.074208+0000","last_change":"2024-04-24T18:11:09.074175+0000","last_active":"2024-04-24T18:11:09.074208+0000","last_peered":"2024-04-24T18:11:09.074208+0000","last_clean":"2024-04-24T18:11:09.074208+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:11:09.074208+0000","last_undegraded":"2024-04-24T18:11:09.074208+0000","last_fullsized":"2024-04-24T18:11:09.074208+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'756","last_scrub_stamp":"2024-04-24T18:11:09.074064+0000","last_deep_scrub":"195'756","last_deep_scrub_stamp":"2024-04-24T18:11:09.074064+0000","last_clean_scrub_stamp":"2024-04-24T18:11:09.074064+0000","objects_scrubbed":73,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:49:14.146958+0000","scrub_duration":0.048926999999999998,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"195'1410","reported_seq":7341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:56.161170+0000","last_change":"2024-04-24T18:10:56.161143+0000","last_active":"2024-04-24T18:10:56.161170+0000","last_peered":"2024-04-24T18:10:56.161170+0000","last_clean":"2024-04-24T18:10:56.161170+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:56.161170+0000","last_undegraded":"2024-04-24T18:10:56.161170+0000","last_fullsized":"2024-04-24T18:10:56.161170+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1410","last_scrub_stamp":"2024-04-24T18:10:56.161075+0000","last_deep_scrub":"195'1410","last_deep_scrub_stamp":"2024-04-24T18:10:56.161075+0000","last_clean_scrub_stamp":"2024-04-24T18:10:56.161075+0000","objects_scrubbed":71,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:46.521665+0000","scrub_duration":0.053384187999999999,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"195'777","reported_seq":6077,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:03.324113+0000","last_change":"2024-04-24T18:11:03.324082+0000","last_active":"2024-04-24T18:11:03.324113+0000","last_peered":"2024-04-24T18:11:03.324113+0000","last_clean":"2024-04-24T18:11:03.324113+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:11:03.324113+0000","last_undegraded":"2024-04-24T18:11:03.324113+0000","last_fullsized":"2024-04-24T18:11:03.324113+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'777","last_scrub_stamp":"2024-04-24T18:11:03.323986+0000","last_deep_scrub":"195'777","last_deep_scrub_stamp":"2024-04-24T18:11:03.323986+0000","last_clean_scrub_stamp":"2024-04-24T18:11:03.323986+0000","objects_scrubbed":67,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:31.514816+0000","scrub_duration":0.052819580999999997,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"195'1126","reported_seq":10323,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.893418+0000","last_change":"2024-04-24T18:10:55.893394+0000","last_active":"2024-04-24T18:10:55.893418+0000","last_peered":"2024-04-24T18:10:55.893418+0000","last_clean":"2024-04-24T18:10:55.893418+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:55.893418+0000","last_undegraded":"2024-04-24T18:10:55.893418+0000","last_fullsized":"2024-04-24T18:10:55.893418+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1126","last_scrub_stamp":"2024-04-24T18:10:55.893313+0000","last_deep_scrub":"195'1126","last_deep_scrub_stamp":"2024-04-24T18:10:55.893313+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.893313+0000","objects_scrubbed":47,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:44.002133+0000","scrub_duration":0.035704430000000002,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"171'535","reported_seq":11657,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:45.033359+0000","last_change":"2024-04-24T18:10:20.332101+0000","last_active":"2024-04-24T18:10:45.033359+0000","last_peered":"2024-04-24T18:10:45.033359+0000","last_clean":"2024-04-24T18:10:45.033359+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:10:45.033359+0000","last_undegraded":"2024-04-24T18:10:45.033359+0000","last_fullsized":"2024-04-24T18:10:45.033359+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":535,"log_dups_size":0,"ondisk_log_size":535,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"195'926","reported_seq":12395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:59.950499+0000","last_change":"2024-04-24T18:10:59.950469+0000","last_active":"2024-04-24T18:10:59.950499+0000","last_peered":"2024-04-24T18:10:59.950499+0000","last_clean":"2024-04-24T18:10:59.950499+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:59.950499+0000","last_undegraded":"2024-04-24T18:10:59.950499+0000","last_fullsized":"2024-04-24T18:10:59.950499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'926","last_scrub_stamp":"2024-04-24T18:10:59.950374+0000","last_deep_scrub":"195'926","last_deep_scrub_stamp":"2024-04-24T18:10:59.950374+0000","last_clean_scrub_stamp":"2024-04-24T18:10:59.950374+0000","objects_scrubbed":46,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:07.614839+0000","scrub_duration":0.035256774999999997,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"167'726","reported_seq":9736,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:45.032791+0000","last_change":"2024-04-24T18:10:20.329954+0000","last_active":"2024-04-24T18:10:45.032791+0000","last_peered":"2024-04-24T18:10:45.032791+0000","last_clean":"2024-04-24T18:10:45.032791+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:10:45.032791+0000","last_undegraded":"2024-04-24T18:10:45.032791+0000","last_fullsized":"2024-04-24T18:10:45.032791+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":726,"log_dups_size":0,"ondisk_log_size":726,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"195'663","reported_seq":11722,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:04.891195+0000","last_change":"2024-04-24T18:11:04.891168+0000","last_active":"2024-04-24T18:11:04.891195+0000","last_peered":"2024-04-24T18:11:04.891195+0000","last_clean":"2024-04-24T18:11:04.891195+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:11:04.891195+0000","last_undegraded":"2024-04-24T18:11:04.891195+0000","last_fullsized":"2024-04-24T18:11:04.891195+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'663","last_scrub_stamp":"2024-04-24T18:11:04.891081+0000","last_deep_scrub":"195'663","last_deep_scrub_stamp":"2024-04-24T18:11:04.891081+0000","last_clean_scrub_stamp":"2024-04-24T18:11:04.891081+0000","objects_scrubbed":47,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:08:27.138817+0000","scrub_duration":0.036953444000000002,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"195'1414","reported_seq":7034,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:12.257755+0000","last_change":"2024-04-24T18:11:12.257729+0000","last_active":"2024-04-24T18:11:12.257755+0000","last_peered":"2024-04-24T18:11:12.257755+0000","last_clean":"2024-04-24T18:11:12.257755+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:11:12.257755+0000","last_undegraded":"2024-04-24T18:11:12.257755+0000","last_fullsized":"2024-04-24T18:11:12.257755+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1414","last_scrub_stamp":"2024-04-24T18:11:12.257627+0000","last_deep_scrub":"195'1414","last_deep_scrub_stamp":"2024-04-24T18:11:12.257627+0000","last_clean_scrub_stamp":"2024-04-24T18:11:12.257627+0000","objects_scrubbed":52,"log_size":1414,"log_dups_size":0,"ondisk_log_size":1414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:24:24.243327+0000","scrub_duration":0.042728743,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"195'1293","reported_seq":5040,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:06.711622+0000","last_change":"2024-04-24T18:11:06.711588+0000","last_active":"2024-04-24T18:11:06.711622+0000","last_peered":"2024-04-24T18:11:06.711622+0000","last_clean":"2024-04-24T18:11:06.711622+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:11:06.711622+0000","last_undegraded":"2024-04-24T18:11:06.711622+0000","last_fullsized":"2024-04-24T18:11:06.711622+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1293","last_scrub_stamp":"2024-04-24T18:11:06.711502+0000","last_deep_scrub":"195'1293","last_deep_scrub_stamp":"2024-04-24T18:11:06.711502+0000","last_clean_scrub_stamp":"2024-04-24T18:11:06.711502+0000","objects_scrubbed":56,"log_size":1293,"log_dups_size":0,"ondisk_log_size":1293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:44:06.596613+0000","scrub_duration":0.058189932,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":170,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:01.052437+0000","last_change":"2024-04-24T18:11:01.052398+0000","last_active":"2024-04-24T18:11:01.052437+0000","last_peered":"2024-04-24T18:11:01.052437+0000","last_clean":"2024-04-24T18:11:01.052437+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:11:01.052437+0000","last_undegraded":"2024-04-24T18:11:01.052437+0000","last_fullsized":"2024-04-24T18:11:01.052437+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:01.052307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:01.052307+0000","last_clean_scrub_stamp":"2024-04-24T18:11:01.052307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:28:46.634682+0000","scrub_duration":0.0020482690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":188,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:58.094646+0000","last_change":"2024-04-24T18:10:58.094616+0000","last_active":"2024-04-24T18:10:58.094646+0000","last_peered":"2024-04-24T18:10:58.094646+0000","last_clean":"2024-04-24T18:10:58.094646+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:58.094646+0000","last_undegraded":"2024-04-24T18:10:58.094646+0000","last_fullsized":"2024-04-24T18:10:58.094646+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:58.094496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:58.094496+0000","last_clean_scrub_stamp":"2024-04-24T18:10:58.094496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:12.046470+0000","scrub_duration":0.0019949719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"195'1013","reported_seq":7249,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:54.187466+0000","last_change":"2024-04-24T18:10:54.187415+0000","last_active":"2024-04-24T18:10:54.187466+0000","last_peered":"2024-04-24T18:10:54.187466+0000","last_clean":"2024-04-24T18:10:54.187466+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:54.187466+0000","last_undegraded":"2024-04-24T18:10:54.187466+0000","last_fullsized":"2024-04-24T18:10:54.187466+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1013","last_scrub_stamp":"2024-04-24T18:10:54.187270+0000","last_deep_scrub":"195'1013","last_deep_scrub_stamp":"2024-04-24T18:10:54.187270+0000","last_clean_scrub_stamp":"2024-04-24T18:10:54.187270+0000","objects_scrubbed":56,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:45:44.666812+0000","scrub_duration":0.051950663000000001,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:57.994744+0000","last_change":"2024-04-24T18:10:57.994715+0000","last_active":"2024-04-24T18:10:57.994744+0000","last_peered":"2024-04-24T18:10:57.994744+0000","last_clean":"2024-04-24T18:10:57.994744+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:57.994744+0000","last_undegraded":"2024-04-24T18:10:57.994744+0000","last_fullsized":"2024-04-24T18:10:57.994744+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:57.994616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:57.994616+0000","last_clean_scrub_stamp":"2024-04-24T18:10:57.994616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:36:43.195928+0000","scrub_duration":0.0019333130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":215,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:58.652707+0000","last_change":"2024-04-24T18:10:58.652656+0000","last_active":"2024-04-24T18:10:58.652707+0000","last_peered":"2024-04-24T18:10:58.652707+0000","last_clean":"2024-04-24T18:10:58.652707+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:58.652707+0000","last_undegraded":"2024-04-24T18:10:58.652707+0000","last_fullsized":"2024-04-24T18:10:58.652707+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:58.652523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:58.652523+0000","last_clean_scrub_stamp":"2024-04-24T18:10:58.652523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:17.862164+0000","scrub_duration":0.002009186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"195'1125","reported_seq":6862,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:57.676169+0000","last_change":"2024-04-24T18:10:57.676133+0000","last_active":"2024-04-24T18:10:57.676169+0000","last_peered":"2024-04-24T18:10:57.676169+0000","last_clean":"2024-04-24T18:10:57.676169+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:57.676169+0000","last_undegraded":"2024-04-24T18:10:57.676169+0000","last_fullsized":"2024-04-24T18:10:57.676169+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1125","last_scrub_stamp":"2024-04-24T18:10:57.676048+0000","last_deep_scrub":"195'1125","last_deep_scrub_stamp":"2024-04-24T18:10:57.676048+0000","last_clean_scrub_stamp":"2024-04-24T18:10:57.676048+0000","objects_scrubbed":64,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:13:51.782347+0000","scrub_duration":0.053100473000000002,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"195'8","reported_seq":199,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.351033+0000","last_change":"2024-04-24T18:10:55.351010+0000","last_active":"2024-04-24T18:10:55.351033+0000","last_peered":"2024-04-24T18:10:55.351033+0000","last_clean":"2024-04-24T18:10:55.351033+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:55.351033+0000","last_undegraded":"2024-04-24T18:10:55.351033+0000","last_fullsized":"2024-04-24T18:10:55.351033+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'8","last_scrub_stamp":"2024-04-24T18:10:55.350928+0000","last_deep_scrub":"195'8","last_deep_scrub_stamp":"2024-04-24T18:10:55.350928+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.350928+0000","objects_scrubbed":4,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:58.630456+0000","scrub_duration":0.006846586,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":170,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":2,"num_bytes_hit_set_archive":170,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.923670+0000","last_change":"2024-04-24T18:10:55.923617+0000","last_active":"2024-04-24T18:10:55.923670+0000","last_peered":"2024-04-24T18:10:55.923670+0000","last_clean":"2024-04-24T18:10:55.923670+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:55.923670+0000","last_undegraded":"2024-04-24T18:10:55.923670+0000","last_fullsized":"2024-04-24T18:10:55.923670+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:55.923513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:55.923513+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.923513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:25:13.453258+0000","scrub_duration":0.001607608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"195'1054","reported_seq":10365,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:00.128949+0000","last_change":"2024-04-24T18:11:00.128923+0000","last_active":"2024-04-24T18:11:00.128949+0000","last_peered":"2024-04-24T18:11:00.128949+0000","last_clean":"2024-04-24T18:11:00.128949+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:11:00.128949+0000","last_undegraded":"2024-04-24T18:11:00.128949+0000","last_fullsized":"2024-04-24T18:11:00.128949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1054","last_scrub_stamp":"2024-04-24T18:11:00.128838+0000","last_deep_scrub":"195'1054","last_deep_scrub_stamp":"2024-04-24T18:11:00.128838+0000","last_clean_scrub_stamp":"2024-04-24T18:11:00.128838+0000","objects_scrubbed":73,"log_size":1054,"log_dups_size":0,"ondisk_log_size":1054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:31.080209+0000","scrub_duration":0.056445503000000001,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":169,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:03.601186+0000","last_change":"2024-04-24T18:11:03.601136+0000","last_active":"2024-04-24T18:11:03.601186+0000","last_peered":"2024-04-24T18:11:03.601186+0000","last_clean":"2024-04-24T18:11:03.601186+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:11:03.601186+0000","last_undegraded":"2024-04-24T18:11:03.601186+0000","last_fullsized":"2024-04-24T18:11:03.601186+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:03.601020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:03.601020+0000","last_clean_scrub_stamp":"2024-04-24T18:11:03.601020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:54.636745+0000","scrub_duration":0.0017346939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:00.947063+0000","last_change":"2024-04-24T18:11:00.947025+0000","last_active":"2024-04-24T18:11:00.947063+0000","last_peered":"2024-04-24T18:11:00.947063+0000","last_clean":"2024-04-24T18:11:00.947063+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:11:00.947063+0000","last_undegraded":"2024-04-24T18:11:00.947063+0000","last_fullsized":"2024-04-24T18:11:00.947063+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:00.946921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:00.946921+0000","last_clean_scrub_stamp":"2024-04-24T18:11:00.946921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:45.644151+0000","scrub_duration":0.0024498359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"195'618","reported_seq":5398,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.570019+0000","last_change":"2024-04-24T18:10:55.569997+0000","last_active":"2024-04-24T18:10:55.570019+0000","last_peered":"2024-04-24T18:10:55.570019+0000","last_clean":"2024-04-24T18:10:55.570019+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:55.570019+0000","last_undegraded":"2024-04-24T18:10:55.570019+0000","last_fullsized":"2024-04-24T18:10:55.570019+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'618","last_scrub_stamp":"2024-04-24T18:10:55.569917+0000","last_deep_scrub":"195'618","last_deep_scrub_stamp":"2024-04-24T18:10:55.569917+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.569917+0000","objects_scrubbed":44,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:26.172108+0000","scrub_duration":0.024712272,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"195'4","reported_seq":53,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:53.116565+0000","last_change":"2024-04-24T18:10:53.116543+0000","last_active":"2024-04-24T18:10:53.116565+0000","last_peered":"2024-04-24T18:10:53.116565+0000","last_clean":"2024-04-24T18:10:53.116565+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:53.116565+0000","last_undegraded":"2024-04-24T18:10:53.116565+0000","last_fullsized":"2024-04-24T18:10:53.116565+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'4","last_scrub_stamp":"2024-04-24T18:10:53.116463+0000","last_deep_scrub":"195'4","last_deep_scrub_stamp":"2024-04-24T18:10:53.116463+0000","last_clean_scrub_stamp":"2024-04-24T18:10:53.116463+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:01.863529+0000","scrub_duration":0.00386317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"195'119","reported_seq":394,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:01.945228+0000","last_change":"2024-04-24T18:11:01.945199+0000","last_active":"2024-04-24T18:11:01.945228+0000","last_peered":"2024-04-24T18:11:01.945228+0000","last_clean":"2024-04-24T18:11:01.945228+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:11:01.945228+0000","last_undegraded":"2024-04-24T18:11:01.945228+0000","last_fullsized":"2024-04-24T18:11:01.945228+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'119","last_scrub_stamp":"2024-04-24T18:11:01.945105+0000","last_deep_scrub":"195'119","last_deep_scrub_stamp":"2024-04-24T18:11:01.945105+0000","last_clean_scrub_stamp":"2024-04-24T18:11:01.945105+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:34.264274+0000","scrub_duration":0.0045221389999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":184,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:08.646322+0000","last_change":"2024-04-24T18:11:08.646293+0000","last_active":"2024-04-24T18:11:08.646322+0000","last_peered":"2024-04-24T18:11:08.646322+0000","last_clean":"2024-04-24T18:11:08.646322+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:11:08.646322+0000","last_undegraded":"2024-04-24T18:11:08.646322+0000","last_fullsized":"2024-04-24T18:11:08.646322+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:08.646187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:08.646187+0000","last_clean_scrub_stamp":"2024-04-24T18:11:08.646187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:14.590728+0000","scrub_duration":0.0021374290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"195'1135","reported_seq":10242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:08.226615+0000","last_change":"2024-04-24T18:11:08.226586+0000","last_active":"2024-04-24T18:11:08.226615+0000","last_peered":"2024-04-24T18:11:08.226615+0000","last_clean":"2024-04-24T18:11:08.226615+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:11:08.226615+0000","last_undegraded":"2024-04-24T18:11:08.226615+0000","last_fullsized":"2024-04-24T18:11:08.226615+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1135","last_scrub_stamp":"2024-04-24T18:11:08.226450+0000","last_deep_scrub":"195'1135","last_deep_scrub_stamp":"2024-04-24T18:11:08.226450+0000","last_clean_scrub_stamp":"2024-04-24T18:11:08.226450+0000","objects_scrubbed":45,"log_size":1135,"log_dups_size":0,"ondisk_log_size":1135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:04.488598+0000","scrub_duration":0.039250445000000002,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"174'671","reported_seq":6167,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:45.033357+0000","last_change":"2024-04-24T18:10:20.332296+0000","last_active":"2024-04-24T18:10:45.033357+0000","last_peered":"2024-04-24T18:10:45.033357+0000","last_clean":"2024-04-24T18:10:45.033357+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:10:45.033357+0000","last_undegraded":"2024-04-24T18:10:45.033357+0000","last_fullsized":"2024-04-24T18:10:45.033357+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":671,"log_dups_size":0,"ondisk_log_size":671,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":194,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:54.390541+0000","last_change":"2024-04-24T18:10:54.390502+0000","last_active":"2024-04-24T18:10:54.390541+0000","last_peered":"2024-04-24T18:10:54.390541+0000","last_clean":"2024-04-24T18:10:54.390541+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:54.390541+0000","last_undegraded":"2024-04-24T18:10:54.390541+0000","last_fullsized":"2024-04-24T18:10:54.390541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:54.390363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:54.390363+0000","last_clean_scrub_stamp":"2024-04-24T18:10:54.390363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:16.769713+0000","scrub_duration":0.0020000529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"195'862","reported_seq":6205,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:59.339106+0000","last_change":"2024-04-24T18:10:59.339045+0000","last_active":"2024-04-24T18:10:59.339106+0000","last_peered":"2024-04-24T18:10:59.339106+0000","last_clean":"2024-04-24T18:10:59.339106+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:59.339106+0000","last_undegraded":"2024-04-24T18:10:59.339106+0000","last_fullsized":"2024-04-24T18:10:59.339106+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'862","last_scrub_stamp":"2024-04-24T18:10:59.338928+0000","last_deep_scrub":"195'862","last_deep_scrub_stamp":"2024-04-24T18:10:59.338928+0000","last_clean_scrub_stamp":"2024-04-24T18:10:59.338928+0000","objects_scrubbed":44,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:28:38.073034+0000","scrub_duration":0.032138798000000003,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":197,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:56.976176+0000","last_change":"2024-04-24T18:10:56.976138+0000","last_active":"2024-04-24T18:10:56.976176+0000","last_peered":"2024-04-24T18:10:56.976176+0000","last_clean":"2024-04-24T18:10:56.976176+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:56.976176+0000","last_undegraded":"2024-04-24T18:10:56.976176+0000","last_fullsized":"2024-04-24T18:10:56.976176+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:56.976030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:56.976030+0000","last_clean_scrub_stamp":"2024-04-24T18:10:56.976030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:11.277547+0000","scrub_duration":0.0015289189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"195'758","reported_seq":12227,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:01.753656+0000","last_change":"2024-04-24T18:11:01.753629+0000","last_active":"2024-04-24T18:11:01.753656+0000","last_peered":"2024-04-24T18:11:01.753656+0000","last_clean":"2024-04-24T18:11:01.753656+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:11:01.753656+0000","last_undegraded":"2024-04-24T18:11:01.753656+0000","last_fullsized":"2024-04-24T18:11:01.753656+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'758","last_scrub_stamp":"2024-04-24T18:11:01.753535+0000","last_deep_scrub":"195'758","last_deep_scrub_stamp":"2024-04-24T18:11:01.753535+0000","last_clean_scrub_stamp":"2024-04-24T18:11:01.753535+0000","objects_scrubbed":62,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:29.522832+0000","scrub_duration":0.052474596999999998,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"171'666","reported_seq":6307,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:45.033152+0000","last_change":"2024-04-24T18:10:20.331718+0000","last_active":"2024-04-24T18:10:45.033152+0000","last_peered":"2024-04-24T18:10:45.033152+0000","last_clean":"2024-04-24T18:10:45.033152+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:10:45.033152+0000","last_undegraded":"2024-04-24T18:10:45.033152+0000","last_fullsized":"2024-04-24T18:10:45.033152+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":666,"log_dups_size":0,"ondisk_log_size":666,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"195'749","reported_seq":4003,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:57.145978+0000","last_change":"2024-04-24T18:10:57.145951+0000","last_active":"2024-04-24T18:10:57.145978+0000","last_peered":"2024-04-24T18:10:57.145978+0000","last_clean":"2024-04-24T18:10:57.145978+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:57.145978+0000","last_undegraded":"2024-04-24T18:10:57.145978+0000","last_fullsized":"2024-04-24T18:10:57.145978+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'749","last_scrub_stamp":"2024-04-24T18:10:57.145859+0000","last_deep_scrub":"195'749","last_deep_scrub_stamp":"2024-04-24T18:10:57.145859+0000","last_clean_scrub_stamp":"2024-04-24T18:10:57.145859+0000","objects_scrubbed":55,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:28.736167+0000","scrub_duration":0.033221462,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"195'730","reported_seq":6373,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.178635+0000","last_change":"2024-04-24T18:10:55.178595+0000","last_active":"2024-04-24T18:10:55.178635+0000","last_peered":"2024-04-24T18:10:55.178635+0000","last_clean":"2024-04-24T18:10:55.178635+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:55.178635+0000","last_undegraded":"2024-04-24T18:10:55.178635+0000","last_fullsized":"2024-04-24T18:10:55.178635+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'730","last_scrub_stamp":"2024-04-24T18:10:55.178457+0000","last_deep_scrub":"195'730","last_deep_scrub_stamp":"2024-04-24T18:10:55.178457+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.178457+0000","objects_scrubbed":48,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:35.889998+0000","scrub_duration":0.033707577000000002,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"195'629","reported_seq":7133,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:06.927423+0000","last_change":"2024-04-24T18:11:06.927387+0000","last_active":"2024-04-24T18:11:06.927423+0000","last_peered":"2024-04-24T18:11:06.927423+0000","last_clean":"2024-04-24T18:11:06.927423+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:11:06.927423+0000","last_undegraded":"2024-04-24T18:11:06.927423+0000","last_fullsized":"2024-04-24T18:11:06.927423+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'629","last_scrub_stamp":"2024-04-24T18:11:06.927275+0000","last_deep_scrub":"195'629","last_deep_scrub_stamp":"2024-04-24T18:11:06.927275+0000","last_clean_scrub_stamp":"2024-04-24T18:11:06.927275+0000","objects_scrubbed":49,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:44.035870+0000","scrub_duration":0.036822510000000003,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"167'634","reported_seq":9281,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:45.033692+0000","last_change":"2024-04-24T18:10:20.334138+0000","last_active":"2024-04-24T18:10:45.033692+0000","last_peered":"2024-04-24T18:10:45.033692+0000","last_clean":"2024-04-24T18:10:45.033692+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:10:45.033692+0000","last_undegraded":"2024-04-24T18:10:45.033692+0000","last_fullsized":"2024-04-24T18:10:45.033692+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":634,"log_dups_size":0,"ondisk_log_size":634,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"167'596","reported_seq":7180,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:45.033756+0000","last_change":"2024-04-24T18:10:20.334800+0000","last_active":"2024-04-24T18:10:45.033756+0000","last_peered":"2024-04-24T18:10:45.033756+0000","last_clean":"2024-04-24T18:10:45.033756+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:10:45.033756+0000","last_undegraded":"2024-04-24T18:10:45.033756+0000","last_fullsized":"2024-04-24T18:10:45.033756+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":596,"log_dups_size":0,"ondisk_log_size":596,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"195'926","reported_seq":6502,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:10.688593+0000","last_change":"2024-04-24T18:11:10.688546+0000","last_active":"2024-04-24T18:11:10.688593+0000","last_peered":"2024-04-24T18:11:10.688593+0000","last_clean":"2024-04-24T18:11:10.688593+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:11:10.688593+0000","last_undegraded":"2024-04-24T18:11:10.688593+0000","last_fullsized":"2024-04-24T18:11:10.688593+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'926","last_scrub_stamp":"2024-04-24T18:11:10.688439+0000","last_deep_scrub":"195'926","last_deep_scrub_stamp":"2024-04-24T18:11:10.688439+0000","last_clean_scrub_stamp":"2024-04-24T18:11:10.688439+0000","objects_scrubbed":58,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:55.416161+0000","scrub_duration":0.048479031999999998,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"167'663","reported_seq":7483,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:45.032609+0000","last_change":"2024-04-24T18:10:20.329149+0000","last_active":"2024-04-24T18:10:45.032609+0000","last_peered":"2024-04-24T18:10:45.032609+0000","last_clean":"2024-04-24T18:10:45.032609+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:10:45.032609+0000","last_undegraded":"2024-04-24T18:10:45.032609+0000","last_fullsized":"2024-04-24T18:10:45.032609+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:07:35.525455+0000","last_clean_scrub_stamp":"2024-04-24T18:07:35.525455+0000","objects_scrubbed":0,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"195'827","reported_seq":9272,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:09.250017+0000","last_change":"2024-04-24T18:11:09.249966+0000","last_active":"2024-04-24T18:11:09.250017+0000","last_peered":"2024-04-24T18:11:09.250017+0000","last_clean":"2024-04-24T18:11:09.250017+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:11:09.250017+0000","last_undegraded":"2024-04-24T18:11:09.250017+0000","last_fullsized":"2024-04-24T18:11:09.250017+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'827","last_scrub_stamp":"2024-04-24T18:11:09.249858+0000","last_deep_scrub":"195'827","last_deep_scrub_stamp":"2024-04-24T18:11:09.249858+0000","last_clean_scrub_stamp":"2024-04-24T18:11:09.249858+0000","objects_scrubbed":62,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:46.516321+0000","scrub_duration":0.049107976999999997,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"195'731","reported_seq":6604,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:56.616201+0000","last_change":"2024-04-24T18:10:56.616175+0000","last_active":"2024-04-24T18:10:56.616201+0000","last_peered":"2024-04-24T18:10:56.616201+0000","last_clean":"2024-04-24T18:10:56.616201+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:56.616201+0000","last_undegraded":"2024-04-24T18:10:56.616201+0000","last_fullsized":"2024-04-24T18:10:56.616201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'731","last_scrub_stamp":"2024-04-24T18:10:56.616086+0000","last_deep_scrub":"195'731","last_deep_scrub_stamp":"2024-04-24T18:10:56.616086+0000","last_clean_scrub_stamp":"2024-04-24T18:10:56.616086+0000","objects_scrubbed":69,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:24.483369+0000","scrub_duration":0.038128871000000002,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"195'1013","reported_seq":9207,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:10.924274+0000","last_change":"2024-04-24T18:11:10.924248+0000","last_active":"2024-04-24T18:11:10.924274+0000","last_peered":"2024-04-24T18:11:10.924274+0000","last_clean":"2024-04-24T18:11:10.924274+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:11:10.924274+0000","last_undegraded":"2024-04-24T18:11:10.924274+0000","last_fullsized":"2024-04-24T18:11:10.924274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1013","last_scrub_stamp":"2024-04-24T18:11:10.924160+0000","last_deep_scrub":"195'1013","last_deep_scrub_stamp":"2024-04-24T18:11:10.924160+0000","last_clean_scrub_stamp":"2024-04-24T18:11:10.924160+0000","objects_scrubbed":46,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:51.640117+0000","scrub_duration":0.036631578999999997,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"195'729","reported_seq":8093,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:08.251570+0000","last_change":"2024-04-24T18:11:08.251543+0000","last_active":"2024-04-24T18:11:08.251570+0000","last_peered":"2024-04-24T18:11:08.251570+0000","last_clean":"2024-04-24T18:11:08.251570+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:11:08.251570+0000","last_undegraded":"2024-04-24T18:11:08.251570+0000","last_fullsized":"2024-04-24T18:11:08.251570+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'729","last_scrub_stamp":"2024-04-24T18:11:08.251456+0000","last_deep_scrub":"195'729","last_deep_scrub_stamp":"2024-04-24T18:11:08.251456+0000","last_clean_scrub_stamp":"2024-04-24T18:11:08.251456+0000","objects_scrubbed":41,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:35:01.655760+0000","scrub_duration":0.034614825000000002,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"195'672","reported_seq":6108,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:02.759003+0000","last_change":"2024-04-24T18:11:02.758968+0000","last_active":"2024-04-24T18:11:02.759003+0000","last_peered":"2024-04-24T18:11:02.759003+0000","last_clean":"2024-04-24T18:11:02.759003+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:11:02.759003+0000","last_undegraded":"2024-04-24T18:11:02.759003+0000","last_fullsized":"2024-04-24T18:11:02.759003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'672","last_scrub_stamp":"2024-04-24T18:11:02.758883+0000","last_deep_scrub":"195'672","last_deep_scrub_stamp":"2024-04-24T18:11:02.758883+0000","last_clean_scrub_stamp":"2024-04-24T18:11:02.758883+0000","objects_scrubbed":51,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:10.975144+0000","scrub_duration":0.042076016000000001,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"195'1226","reported_seq":7085,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:58.193853+0000","last_change":"2024-04-24T18:10:58.193822+0000","last_active":"2024-04-24T18:10:58.193853+0000","last_peered":"2024-04-24T18:10:58.193853+0000","last_clean":"2024-04-24T18:10:58.193853+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:58.193853+0000","last_undegraded":"2024-04-24T18:10:58.193853+0000","last_fullsized":"2024-04-24T18:10:58.193853+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1226","last_scrub_stamp":"2024-04-24T18:10:58.193720+0000","last_deep_scrub":"195'1226","last_deep_scrub_stamp":"2024-04-24T18:10:58.193720+0000","last_clean_scrub_stamp":"2024-04-24T18:10:58.193720+0000","objects_scrubbed":63,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:15.346904+0000","scrub_duration":0.051315212999999998,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":27653,"ondisk_log_size":27653,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":253,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":253,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574899,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307900,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063940,"statfs":{"total":96636764160,"available":96321474560,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.47999999999999998,"15min":0.46600000000000003},"min":{"1min":0.253,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.53000000000000003,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.501},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.48599999999999999,"15min":0.47299999999999998},"min":{"1min":0.249,"5min":0.221,"15min":0.221},"max":{"1min":0.67000000000000004,"5min":2.585,"15min":2.585},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.54400000000000004,"15min":0.5},"min":{"1min":0.26400000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.78300000000000003,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46600000000000003,"15min":0.45600000000000002},"min":{"1min":0.29599999999999999,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63400000000000001,"5min":2.008,"15min":2.008},"last":0.47499999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.47699999999999998,"5min":0.55100000000000005,"15min":0.51000000000000001},"min":{"1min":0.27700000000000002,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.64200000000000002,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.54000000000000004,"15min":0.501},"min":{"1min":0.38700000000000001,"5min":0.307,"15min":0.307},"max":{"1min":0.71199999999999997,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.57099999999999995}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.53700000000000003,"15min":0.51600000000000001},"min":{"1min":0.42999999999999999,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.753,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.51900000000000002,"15min":0.52100000000000002},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.80400000000000005,"5min":1.196,"15min":1.196},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.45000000000000001,"15min":0.443},"min":{"1min":0.29799999999999999,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.77100000000000002,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.495,"15min":0.501},"min":{"1min":0.34399999999999997,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.69499999999999995,"5min":1.131,"15min":1.131},"last":0.374}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.499,"15min":0.499},"min":{"1min":0.318,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.77900000000000003,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.53800000000000003,"15min":0.53700000000000003},"min":{"1min":0.36199999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.73999999999999999,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.40500000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72899999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.436},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.998,"15min":0.998},"last":0.39400000000000002}]}]},{"osd":6,"up_from":95,"seq":408021893150,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313220,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":310880,"kb_avail":94058620,"statfs":{"total":96636764160,"available":96316026880,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":318341151},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.439},"min":{"1min":0.33000000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89100000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.53000000000000003}]},{"osd":1,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.874,"15min":0.92600000000000005},"min":{"1min":0.41099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.96099999999999997,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69099999999999995,"5min":1.26,"15min":1.26},"last":0.59099999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61099999999999999,"15min":0.61699999999999999},"min":{"1min":0.42099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.94899999999999995,"5min":2.306,"15min":2.306},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78100000000000003,"5min":2.468,"15min":2.468},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79600000000000004,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.60799999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.42399999999999999}]},{"osd":5,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71999999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.39400000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81200000000000006,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":1.115}]},{"osd":7,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81100000000000005,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.51100000000000001,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.46899999999999997}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307884,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063956,"statfs":{"total":96636764160,"available":96321490944,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.33500000000000002,"5min":0.53500000000000003,"15min":0.61799999999999999},"min":{"1min":0.20899999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.5,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.36799999999999999,"5min":0.89500000000000002,"15min":1.127},"min":{"1min":0.23799999999999999,"5min":0.191,"15min":0.191},"max":{"1min":0.55600000000000005,"5min":14.15,"15min":14.15},"last":0.34200000000000003}]},{"osd":2,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.47799999999999998,"15min":0.498},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.93600000000000005,"5min":1.306,"15min":1.306},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.433,"5min":0.96099999999999997,"15min":1.1879999999999999},"min":{"1min":0.28899999999999998,"5min":0.217,"15min":0.217},"max":{"1min":0.95499999999999996,"5min":14.118,"15min":14.118},"last":0.45300000000000001}]},{"osd":3,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.93000000000000005,"15min":1.1519999999999999},"min":{"1min":0.26600000000000001,"5min":0.248,"15min":0.248},"max":{"1min":0.67600000000000005,"5min":14.089,"15min":14.089},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.59099999999999997,"15min":0.67600000000000005},"min":{"1min":0.252,"5min":0.248,"15min":0.248},"max":{"1min":0.59799999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.51000000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.57599999999999996,"15min":0.61099999999999999},"min":{"1min":0.32000000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71799999999999997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.53500000000000003,"5min":1.0600000000000001,"15min":1.274},"min":{"1min":0.35699999999999998,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.73299999999999998,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.66700000000000004}]},{"osd":5,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.68700000000000006,"15min":0.76300000000000001},"min":{"1min":0.371,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":1.2310000000000001,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.63300000000000001,"15min":0.66800000000000004},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.746,"5min":1.49,"15min":1.49},"last":0.59299999999999997}]},{"osd":6,"last update":"Wed Apr 24 18:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55200000000000005,"15min":0.53900000000000003},"min":{"1min":0.436,"5min":0.379,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.57699999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:10:21 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":1.073,"15min":1.2929999999999999},"min":{"1min":0.438,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.1279999999999999,"5min":14.051,"15min":14.051},"last":0.622},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.74399999999999999,"15min":0.82299999999999995},"min":{"1min":0.42899999999999999,"5min":0.373,"15min":0.373},"max":{"1min":1.1870000000000001,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.65300000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574899,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311944,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059896,"statfs":{"total":96636764160,"available":96317333504,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.40000000000000002,"15min":0.41799999999999998},"min":{"1min":0.25900000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.65700000000000003,"5min":1.375,"15min":1.375},"last":0.28599999999999998},{"interface":"front","average":{"1min":0.41299999999999998,"5min":1.2549999999999999,"15min":1.6379999999999999},"min":{"1min":0.23699999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":0.90200000000000002,"5min":26.936,"15min":26.936},"last":0.39000000000000001}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.92000000000000004,"15min":1.1479999999999999},"min":{"1min":0.23400000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.61699999999999999,"5min":11.93,"15min":11.93},"last":0.41199999999999998},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.42699999999999999,"15min":0.36199999999999999},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":1.157,"5min":6.4889999999999999,"15min":6.4889999999999999},"last":0.34499999999999997}]},{"osd":3,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.45800000000000002,"5min":0.83599999999999997,"15min":0.995},"min":{"1min":0.25900000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.93799999999999994,"5min":14.371,"15min":14.371},"last":0.36099999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.86699999999999999,"15min":0.97399999999999998},"min":{"1min":0.311,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.71499999999999997,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.55900000000000005,"5min":0.92800000000000005,"15min":1.1140000000000001},"min":{"1min":0.375,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96799999999999997,"5min":14.414,"15min":14.414},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56299999999999994,"15min":0.57499999999999996},"min":{"1min":0.35399999999999998,"5min":0.317,"15min":0.317},"max":{"1min":0.93200000000000005,"5min":1.329,"15min":1.329},"last":0.54100000000000004}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.73099999999999998,"15min":0.76800000000000002},"min":{"1min":0.37,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.83099999999999996,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.97099999999999997,"15min":1.133},"min":{"1min":0.441,"5min":0.379,"15min":0.379},"max":{"1min":0.995,"5min":14.452,"15min":14.452},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Apr 24 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.47499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88700000000000001,"5min":1.048,"15min":1.048},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.76600000000000001,"15min":0.80200000000000005},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.86099999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.628,"5min":0.98699999999999999,"15min":1.1479999999999999},"min":{"1min":0.42799999999999999,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.1259999999999999,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.59199999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574898,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.33900000000000002,"5min":0.42899999999999999,"15min":0.41499999999999998},"min":{"1min":0.223,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.47499999999999998,"5min":4.109,"15min":4.109},"last":0.315},{"interface":"front","average":{"1min":0.38800000000000001,"5min":0.441,"15min":0.42099999999999999},"min":{"1min":0.20100000000000001,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.77900000000000003,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.34399999999999997,"5min":0.434,"15min":0.40699999999999997},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.55500000000000005,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.39000000000000001,"5min":0.40500000000000003,"15min":0.39300000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.64200000000000002,"5min":1.619,"15min":1.619},"last":0.379}]},{"osd":3,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.40400000000000003,"5min":0.42699999999999999,"15min":0.42799999999999999},"min":{"1min":0.312,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52100000000000002,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.52400000000000002,"15min":0.48599999999999999},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.65000000000000002,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.52500000000000002}]},{"osd":4,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.60299999999999998,"15min":0.57499999999999996},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.66100000000000003,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.54800000000000004,"15min":0.56599999999999995},"min":{"1min":0.38500000000000001,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.80300000000000005,"5min":1.155,"15min":1.155},"last":0.443}]},{"osd":5,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.54400000000000004,"5min":0.59499999999999997,"15min":0.59299999999999997},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.78600000000000003,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":0.84899999999999998,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.54700000000000004}]},{"osd":6,"last update":"Wed Apr 24 18:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.99099999999999999,"15min":1.0469999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.006,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.56499999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:10:18 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.71799999999999997,"15min":0.65600000000000003},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.754,"5min":10.516,"15min":10.516},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.623,"15min":0.61799999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.82699999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.625}]}]},{"osd":3,"up_from":13,"seq":55834574899,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2367488,"data_stored":1201291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.36299999999999999,"15min":0.35599999999999998},"min":{"1min":0.23000000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":0.41799999999999998,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.34000000000000002,"15min":0.33200000000000002},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.497,"5min":0.58699999999999997,"15min":0.58699999999999997},"last":0.44800000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.36799999999999999,"5min":0.40400000000000003,"15min":0.42199999999999999},"min":{"1min":0.23699999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.48099999999999998,"5min":1.792,"15min":1.792},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.41699999999999998,"15min":0.40799999999999997},"min":{"1min":0.28499999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.48499999999999999,"5min":2.8999999999999999,"15min":2.8999999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.38600000000000001,"5min":0.38300000000000001,"15min":0.377},"min":{"1min":0.28799999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.51700000000000002,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.432},{"interface":"front","average":{"1min":0.41899999999999998,"5min":0.46899999999999997,"15min":0.44800000000000001},"min":{"1min":0.32300000000000001,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.50900000000000001,"5min":4.6699999999999999,"15min":4.6699999999999999},"last":0.58899999999999997}]},{"osd":4,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.56699999999999995,"15min":0.56100000000000005},"min":{"1min":0.36499999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.68400000000000005,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53600000000000003,"15min":0.53700000000000003},"min":{"1min":0.33600000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.76300000000000001,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.58399999999999996,"15min":0.58099999999999996},"min":{"1min":0.38300000000000001,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.77700000000000002,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.59299999999999997,"15min":0.58699999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.70699999999999996,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.54800000000000004}]},{"osd":6,"last update":"Wed Apr 24 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.282,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.76900000000000002},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93700000000000006,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.57999999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:10:23 2024","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.624,"15min":0.59499999999999997},"min":{"1min":0.437,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.755,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.74299999999999999,"15min":0.65400000000000003},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.84299999999999997,"5min":15.4,"15min":15.4},"last":0.876}]}]},{"osd":4,"up_from":13,"seq":55834574899,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311948,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059892,"statfs":{"total":96636764160,"available":96317329408,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.44500000000000001,"5min":0.46800000000000003,"15min":0.47499999999999998},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.498,"5min":0.495,"15min":0.497},"min":{"1min":0.38900000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.66000000000000003,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.59799999999999998}]},{"osd":1,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42999999999999999,"5min":0.47699999999999998,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.55000000000000004,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.628},{"interface":"front","average":{"1min":0.45800000000000002,"5min":0.442,"15min":0.45100000000000001},"min":{"1min":0.255,"5min":0.251,"15min":0.251},"max":{"1min":0.626,"5min":0.627,"15min":0.627},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.45300000000000001,"15min":0.45600000000000002},"min":{"1min":0.33900000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.65400000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.48599999999999999,"15min":0.49399999999999999},"min":{"1min":0.27600000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.63800000000000001,"5min":1.153,"15min":1.153},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.34799999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67000000000000004,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.47699999999999998,"5min":0.51800000000000002,"15min":0.503},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.63,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.39800000000000002,"5min":0.41799999999999998,"15min":0.42399999999999999},"min":{"1min":0.28399999999999997,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.628,"5min":1.373,"15min":1.373},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.42299999999999999,"5min":0.435,"15min":0.44700000000000001},"min":{"1min":0.29499999999999998,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.63200000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.42899999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.40600000000000003},"min":{"1min":0.29699999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.39400000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.46600000000000003},"min":{"1min":0.318,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.50600000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:10:25 2024","interfaces":[{"interface":"back","average":{"1min":0.47299999999999998,"5min":0.48999999999999999,"15min":0.50700000000000001},"min":{"1min":0.28000000000000003,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.71099999999999997,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.40699999999999997},{"interface":"front","average":{"1min":0.503,"5min":0.52600000000000002,"15min":0.54300000000000004},"min":{"1min":0.34000000000000002,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.82499999999999996,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.45000000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574899,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307852,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063988,"statfs":{"total":96636764160,"available":96321523712,"internally_reserved":0,"allocated":2363392,"data_stored":1201272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.43099999999999999,"15min":0.433},"min":{"1min":0.307,"5min":0.189,"15min":0.189},"max":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.22500000000000001,"5min":0.222,"15min":0.222},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.38}]},{"osd":1,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.45300000000000001,"15min":0.44600000000000001},"min":{"1min":0.35399999999999998,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":0.89800000000000002},"last":0.501},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.48699999999999999,"15min":0.495},"min":{"1min":0.29799999999999999,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.81299999999999994,"5min":0.84899999999999998,"15min":0.84899999999999998},"last":0.64400000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.48399999999999999,"5min":0.51100000000000001,"15min":0.53900000000000003},"min":{"1min":0.34599999999999997,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.872,"5min":1.296,"15min":1.296},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.44,"5min":0.504,"15min":0.53100000000000003},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.63300000000000001,"5min":1.377,"15min":1.377},"last":0.54200000000000004}]},{"osd":3,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.58999999999999997,"15min":0.63600000000000001},"min":{"1min":0.35999999999999999,"5min":0.313,"15min":0.313},"max":{"1min":0.65200000000000002,"5min":1.899,"15min":1.899},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.58899999999999997,"15min":0.63700000000000001},"min":{"1min":0.307,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.59899999999999998,"5min":1.881,"15min":1.881},"last":0.48599999999999999}]},{"osd":4,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.46999999999999997,"15min":0.503},"min":{"1min":0.27400000000000002,"5min":0.25,"15min":0.25},"max":{"1min":0.66100000000000003,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.442},{"interface":"front","average":{"1min":0.45600000000000002,"5min":0.52000000000000002,"15min":0.56000000000000005},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.80400000000000005,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.53100000000000003}]},{"osd":6,"last update":"Wed Apr 24 18:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.34599999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Apr 24 18:10:22 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.54100000000000004,"15min":0.57699999999999996},"min":{"1min":0.35399999999999998,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.68899999999999995,"5min":1.016,"15min":1.016},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.54400000000000004,"15min":0.58199999999999996},"min":{"1min":0.33700000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.67800000000000005,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.621}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":170,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:11:16.590 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:11:16.590 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:11:16.590 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:11:16.590 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:11:16.591 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:11:16.591 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:11:16.591 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-24T18:07:35.525455+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=7, tm_sec=35, tm_wday=2, tm_yday=115, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=24, tm_hour=18, tm_min=10, tm_sec=50, tm_wday=2, tm_yday=115, tm_isdst=0) 2024-04-24T18:11:16.591 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-24T18:11:36.592 DEBUG:teuthology.orchestra.run.smithi049:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-24T18:11:36.939 INFO:teuthology.orchestra.run.smithi049.stdout: 2024-04-24T18:11:36.940 INFO:teuthology.orchestra.run.smithi049.stderr:dumped all 2024-04-24T18:11:36.951 INFO:teuthology.orchestra.run.smithi049.stdout:{"pg_ready":true,"pg_map":{"version":317,"stamp":"2024-04-24T18:11:35.348553+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590725,"num_objects":1801,"num_object_clones":0,"num_object_copies":3602,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1794,"num_whiteouts":1791,"num_read":292907,"num_read_kb":861674,"num_write":29265,"num_write_kb":534158,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2902,"num_bytes_recovered":46776131,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":338,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":28188,"ondisk_log_size":28188,"up":90,"acting":90,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":90,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2472360,"kb_used_data":18536,"kb_used_omap":43,"kb_used_meta":2453716,"kb_avail":752502360,"statfs":{"total":773094113280,"available":770562416640,"internally_reserved":0,"allocated":18980864,"data_stored":10790266,"data_compressed":18366,"data_compressed_allocated":73728,"data_compressed_original":1163264,"omap_allocated":44665,"internal_metadata":2512605575},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002889"},"pg_stats":[{"pgid":"4.1f","version":"195'756","reported_seq":7316,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:09.074208+0000","last_change":"2024-04-24T18:11:09.074175+0000","last_active":"2024-04-24T18:11:09.074208+0000","last_peered":"2024-04-24T18:11:09.074208+0000","last_clean":"2024-04-24T18:11:09.074208+0000","last_became_active":"2024-04-24T18:10:27.325625+0000","last_became_peered":"2024-04-24T18:10:27.325625+0000","last_unstale":"2024-04-24T18:11:09.074208+0000","last_undegraded":"2024-04-24T18:11:09.074208+0000","last_fullsized":"2024-04-24T18:11:09.074208+0000","mapping_epoch":178,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":179,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'756","last_scrub_stamp":"2024-04-24T18:11:09.074064+0000","last_deep_scrub":"195'756","last_deep_scrub_stamp":"2024-04-24T18:11:09.074064+0000","last_clean_scrub_stamp":"2024-04-24T18:11:09.074064+0000","objects_scrubbed":73,"log_size":756,"log_dups_size":0,"ondisk_log_size":756,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:49:14.146958+0000","scrub_duration":0.048926999999999998,"objects_trimmed":0,"snaptrim_duration":0.00049604300000000005,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":7246,"num_read_kb":29261,"num_write":621,"num_write_kb":11963,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":498,"num_bytes_recovered":8316376,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"195'1410","reported_seq":7341,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:56.161170+0000","last_change":"2024-04-24T18:10:56.161143+0000","last_active":"2024-04-24T18:10:56.161170+0000","last_peered":"2024-04-24T18:10:56.161170+0000","last_clean":"2024-04-24T18:10:56.161170+0000","last_became_active":"2024-04-24T18:10:21.330525+0000","last_became_peered":"2024-04-24T18:10:21.330525+0000","last_unstale":"2024-04-24T18:10:56.161170+0000","last_undegraded":"2024-04-24T18:10:56.161170+0000","last_fullsized":"2024-04-24T18:10:56.161170+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1410","last_scrub_stamp":"2024-04-24T18:10:56.161075+0000","last_deep_scrub":"195'1410","last_deep_scrub_stamp":"2024-04-24T18:10:56.161075+0000","last_clean_scrub_stamp":"2024-04-24T18:10:56.161075+0000","objects_scrubbed":71,"log_size":1410,"log_dups_size":0,"ondisk_log_size":1410,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:39:46.521665+0000","scrub_duration":0.053384187999999999,"objects_trimmed":0,"snaptrim_duration":0.0010775870000000001,"stat_sum":{"num_bytes":0,"num_objects":71,"num_object_clones":0,"num_object_copies":142,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":71,"num_whiteouts":71,"num_read":10332,"num_read_kb":26108,"num_write":1882,"num_write_kb":10825,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":211,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1d","version":"195'777","reported_seq":6077,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:03.324113+0000","last_change":"2024-04-24T18:11:03.324082+0000","last_active":"2024-04-24T18:11:03.324113+0000","last_peered":"2024-04-24T18:11:03.324113+0000","last_clean":"2024-04-24T18:11:03.324113+0000","last_became_active":"2024-04-24T18:07:56.631919+0000","last_became_peered":"2024-04-24T18:07:56.631919+0000","last_unstale":"2024-04-24T18:11:03.324113+0000","last_undegraded":"2024-04-24T18:11:03.324113+0000","last_fullsized":"2024-04-24T18:11:03.324113+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'777","last_scrub_stamp":"2024-04-24T18:11:03.323986+0000","last_deep_scrub":"195'777","last_deep_scrub_stamp":"2024-04-24T18:11:03.323986+0000","last_clean_scrub_stamp":"2024-04-24T18:11:03.323986+0000","objects_scrubbed":67,"log_size":777,"log_dups_size":0,"ondisk_log_size":777,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:19:31.514816+0000","scrub_duration":0.052819580999999997,"objects_trimmed":0,"snaptrim_duration":0.001593924,"stat_sum":{"num_bytes":0,"num_objects":67,"num_object_clones":0,"num_object_copies":134,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":67,"num_whiteouts":67,"num_read":6263,"num_read_kb":18013,"num_write":623,"num_write_kb":12156,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":19,"num_bytes_recovered":1640168,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"195'1126","reported_seq":10323,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.893418+0000","last_change":"2024-04-24T18:10:55.893394+0000","last_active":"2024-04-24T18:10:55.893418+0000","last_peered":"2024-04-24T18:10:55.893418+0000","last_clean":"2024-04-24T18:10:55.893418+0000","last_became_active":"2024-04-24T18:07:56.634571+0000","last_became_peered":"2024-04-24T18:07:56.634571+0000","last_unstale":"2024-04-24T18:10:55.893418+0000","last_undegraded":"2024-04-24T18:10:55.893418+0000","last_fullsized":"2024-04-24T18:10:55.893418+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1126","last_scrub_stamp":"2024-04-24T18:10:55.893313+0000","last_deep_scrub":"195'1126","last_deep_scrub_stamp":"2024-04-24T18:10:55.893313+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.893313+0000","objects_scrubbed":47,"log_size":1126,"log_dups_size":0,"ondisk_log_size":1126,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:45:44.002133+0000","scrub_duration":0.035704430000000002,"objects_trimmed":0,"snaptrim_duration":0.00063761000000000004,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":46,"num_read":12194,"num_read_kb":23610,"num_write":1392,"num_write_kb":9993,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1642727,"num_keys_recovered":20,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1b","version":"195'581","reported_seq":11714,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:29.006118+0000","last_change":"2024-04-24T18:11:29.006090+0000","last_active":"2024-04-24T18:11:29.006118+0000","last_peered":"2024-04-24T18:11:29.006118+0000","last_clean":"2024-04-24T18:11:29.006118+0000","last_became_active":"2024-04-24T18:07:36.551626+0000","last_became_peered":"2024-04-24T18:07:36.551626+0000","last_unstale":"2024-04-24T18:11:29.006118+0000","last_undegraded":"2024-04-24T18:11:29.006118+0000","last_fullsized":"2024-04-24T18:11:29.006118+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'581","last_scrub_stamp":"2024-04-24T18:11:29.006004+0000","last_deep_scrub":"195'581","last_deep_scrub_stamp":"2024-04-24T18:11:29.006004+0000","last_clean_scrub_stamp":"2024-04-24T18:11:29.006004+0000","objects_scrubbed":46,"log_size":581,"log_dups_size":0,"ondisk_log_size":581,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:42:26.208115+0000","scrub_duration":0.035046141000000003,"objects_trimmed":0,"snaptrim_duration":0.001377676,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10925,"num_read_kb":37139,"num_write":483,"num_write_kb":19175,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1a","version":"195'926","reported_seq":12395,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:59.950499+0000","last_change":"2024-04-24T18:10:59.950469+0000","last_active":"2024-04-24T18:10:59.950499+0000","last_peered":"2024-04-24T18:10:59.950499+0000","last_clean":"2024-04-24T18:10:59.950499+0000","last_became_active":"2024-04-24T18:07:36.551337+0000","last_became_peered":"2024-04-24T18:07:36.551337+0000","last_unstale":"2024-04-24T18:10:59.950499+0000","last_undegraded":"2024-04-24T18:10:59.950499+0000","last_fullsized":"2024-04-24T18:10:59.950499+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'926","last_scrub_stamp":"2024-04-24T18:10:59.950374+0000","last_deep_scrub":"195'926","last_deep_scrub_stamp":"2024-04-24T18:10:59.950374+0000","last_clean_scrub_stamp":"2024-04-24T18:10:59.950374+0000","objects_scrubbed":46,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:06:07.614839+0000","scrub_duration":0.035256774999999997,"objects_trimmed":0,"snaptrim_duration":0.001504455,"stat_sum":{"num_bytes":19,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":45,"num_read":14240,"num_read_kb":30482,"num_write":985,"num_write_kb":12171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"195'788","reported_seq":9810,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:13.283987+0000","last_change":"2024-04-24T18:11:13.283960+0000","last_active":"2024-04-24T18:11:13.283987+0000","last_peered":"2024-04-24T18:11:13.283987+0000","last_clean":"2024-04-24T18:11:13.283987+0000","last_became_active":"2024-04-24T18:07:36.550227+0000","last_became_peered":"2024-04-24T18:07:36.550227+0000","last_unstale":"2024-04-24T18:11:13.283987+0000","last_undegraded":"2024-04-24T18:11:13.283987+0000","last_fullsized":"2024-04-24T18:11:13.283987+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'788","last_scrub_stamp":"2024-04-24T18:11:13.283861+0000","last_deep_scrub":"195'788","last_deep_scrub_stamp":"2024-04-24T18:11:13.283861+0000","last_clean_scrub_stamp":"2024-04-24T18:11:13.283861+0000","objects_scrubbed":62,"log_size":788,"log_dups_size":0,"ondisk_log_size":788,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:38:11.289258+0000","scrub_duration":0.047235208000000001,"objects_trimmed":0,"snaptrim_duration":0.0015223540000000001,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":10223,"num_read_kb":31215,"num_write":808,"num_write_kb":15858,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"195'663","reported_seq":11722,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:04.891195+0000","last_change":"2024-04-24T18:11:04.891168+0000","last_active":"2024-04-24T18:11:04.891195+0000","last_peered":"2024-04-24T18:11:04.891195+0000","last_clean":"2024-04-24T18:11:04.891195+0000","last_became_active":"2024-04-24T18:08:57.903197+0000","last_became_peered":"2024-04-24T18:08:57.903197+0000","last_unstale":"2024-04-24T18:11:04.891195+0000","last_undegraded":"2024-04-24T18:11:04.891195+0000","last_fullsized":"2024-04-24T18:11:04.891195+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'663","last_scrub_stamp":"2024-04-24T18:11:04.891081+0000","last_deep_scrub":"195'663","last_deep_scrub_stamp":"2024-04-24T18:11:04.891081+0000","last_clean_scrub_stamp":"2024-04-24T18:11:04.891081+0000","objects_scrubbed":47,"log_size":663,"log_dups_size":0,"ondisk_log_size":663,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:08:27.138817+0000","scrub_duration":0.036953444000000002,"objects_trimmed":0,"snaptrim_duration":0.0032291049999999999,"stat_sum":{"num_bytes":0,"num_objects":47,"num_object_clones":0,"num_object_copies":94,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":47,"num_whiteouts":47,"num_read":12333,"num_read_kb":36289,"num_write":588,"num_write_kb":30745,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":86,"num_bytes_recovered":2524163,"num_keys_recovered":23,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.17","version":"195'1414","reported_seq":7034,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:12.257755+0000","last_change":"2024-04-24T18:11:12.257729+0000","last_active":"2024-04-24T18:11:12.257755+0000","last_peered":"2024-04-24T18:11:12.257755+0000","last_clean":"2024-04-24T18:11:12.257755+0000","last_became_active":"2024-04-24T18:07:36.551328+0000","last_became_peered":"2024-04-24T18:07:36.551328+0000","last_unstale":"2024-04-24T18:11:12.257755+0000","last_undegraded":"2024-04-24T18:11:12.257755+0000","last_fullsized":"2024-04-24T18:11:12.257755+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1414","last_scrub_stamp":"2024-04-24T18:11:12.257627+0000","last_deep_scrub":"195'1414","last_deep_scrub_stamp":"2024-04-24T18:11:12.257627+0000","last_clean_scrub_stamp":"2024-04-24T18:11:12.257627+0000","objects_scrubbed":52,"log_size":1414,"log_dups_size":0,"ondisk_log_size":1414,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:24:24.243327+0000","scrub_duration":0.042728743,"objects_trimmed":0,"snaptrim_duration":0.0012728640000000001,"stat_sum":{"num_bytes":0,"num_objects":52,"num_object_clones":0,"num_object_copies":104,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":52,"num_whiteouts":52,"num_read":10556,"num_read_kb":29373,"num_write":2071,"num_write_kb":19699,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2","version":"195'1293","reported_seq":5040,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:06.711622+0000","last_change":"2024-04-24T18:11:06.711588+0000","last_active":"2024-04-24T18:11:06.711622+0000","last_peered":"2024-04-24T18:11:06.711622+0000","last_clean":"2024-04-24T18:11:06.711622+0000","last_became_active":"2024-04-24T18:07:56.628578+0000","last_became_peered":"2024-04-24T18:07:56.628578+0000","last_unstale":"2024-04-24T18:11:06.711622+0000","last_undegraded":"2024-04-24T18:11:06.711622+0000","last_fullsized":"2024-04-24T18:11:06.711622+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1293","last_scrub_stamp":"2024-04-24T18:11:06.711502+0000","last_deep_scrub":"195'1293","last_deep_scrub_stamp":"2024-04-24T18:11:06.711502+0000","last_clean_scrub_stamp":"2024-04-24T18:11:06.711502+0000","objects_scrubbed":56,"log_size":1293,"log_dups_size":0,"ondisk_log_size":1293,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:44:06.596613+0000","scrub_duration":0.058189932,"objects_trimmed":0,"snaptrim_duration":0.00056209100000000002,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":56,"num_read":8644,"num_read_kb":24165,"num_write":1711,"num_write_kb":17601,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":650878,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.7","version":"0'0","reported_seq":170,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:01.052437+0000","last_change":"2024-04-24T18:11:01.052398+0000","last_active":"2024-04-24T18:11:01.052437+0000","last_peered":"2024-04-24T18:11:01.052437+0000","last_clean":"2024-04-24T18:11:01.052437+0000","last_became_active":"2024-04-24T18:08:53.874304+0000","last_became_peered":"2024-04-24T18:08:53.874304+0000","last_unstale":"2024-04-24T18:11:01.052437+0000","last_undegraded":"2024-04-24T18:11:01.052437+0000","last_fullsized":"2024-04-24T18:11:01.052437+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:01.052307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:01.052307+0000","last_clean_scrub_stamp":"2024-04-24T18:11:01.052307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:28:46.634682+0000","scrub_duration":0.0020482690000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":188,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:58.094646+0000","last_change":"2024-04-24T18:10:58.094616+0000","last_active":"2024-04-24T18:10:58.094646+0000","last_peered":"2024-04-24T18:10:58.094646+0000","last_clean":"2024-04-24T18:10:58.094646+0000","last_became_active":"2024-04-24T18:10:18.331550+0000","last_became_peered":"2024-04-24T18:10:18.331550+0000","last_unstale":"2024-04-24T18:10:58.094646+0000","last_undegraded":"2024-04-24T18:10:58.094646+0000","last_fullsized":"2024-04-24T18:10:58.094646+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:58.094496+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:58.094496+0000","last_clean_scrub_stamp":"2024-04-24T18:10:58.094496+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:12.046470+0000","scrub_duration":0.0019949719999999998,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"195'1013","reported_seq":7249,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:54.187466+0000","last_change":"2024-04-24T18:10:54.187415+0000","last_active":"2024-04-24T18:10:54.187466+0000","last_peered":"2024-04-24T18:10:54.187466+0000","last_clean":"2024-04-24T18:10:54.187466+0000","last_became_active":"2024-04-24T18:10:21.331281+0000","last_became_peered":"2024-04-24T18:10:21.331281+0000","last_unstale":"2024-04-24T18:10:54.187466+0000","last_undegraded":"2024-04-24T18:10:54.187466+0000","last_fullsized":"2024-04-24T18:10:54.187466+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1013","last_scrub_stamp":"2024-04-24T18:10:54.187270+0000","last_deep_scrub":"195'1013","last_deep_scrub_stamp":"2024-04-24T18:10:54.187270+0000","last_clean_scrub_stamp":"2024-04-24T18:10:54.187270+0000","objects_scrubbed":56,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:45:44.666812+0000","scrub_duration":0.051950663000000001,"objects_trimmed":0,"snaptrim_duration":0.00092551899999999997,"stat_sum":{"num_bytes":0,"num_objects":56,"num_object_clones":0,"num_object_copies":112,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":56,"num_whiteouts":55,"num_read":7649,"num_read_kb":32281,"num_write":936,"num_write_kb":18839,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":372,"num_bytes_recovered":138078,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:57.994744+0000","last_change":"2024-04-24T18:10:57.994715+0000","last_active":"2024-04-24T18:10:57.994744+0000","last_peered":"2024-04-24T18:10:57.994744+0000","last_clean":"2024-04-24T18:10:57.994744+0000","last_became_active":"2024-04-24T18:10:18.329769+0000","last_became_peered":"2024-04-24T18:10:18.329769+0000","last_unstale":"2024-04-24T18:10:57.994744+0000","last_undegraded":"2024-04-24T18:10:57.994744+0000","last_fullsized":"2024-04-24T18:10:57.994744+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:57.994616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:57.994616+0000","last_clean_scrub_stamp":"2024-04-24T18:10:57.994616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:36:43.195928+0000","scrub_duration":0.0019333130000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":215,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:58.652707+0000","last_change":"2024-04-24T18:10:58.652656+0000","last_active":"2024-04-24T18:10:58.652707+0000","last_peered":"2024-04-24T18:10:58.652707+0000","last_clean":"2024-04-24T18:10:58.652707+0000","last_became_active":"2024-04-24T18:10:18.328117+0000","last_became_peered":"2024-04-24T18:10:18.328117+0000","last_unstale":"2024-04-24T18:10:58.652707+0000","last_undegraded":"2024-04-24T18:10:58.652707+0000","last_fullsized":"2024-04-24T18:10:58.652707+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:58.652523+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:58.652523+0000","last_clean_scrub_stamp":"2024-04-24T18:10:58.652523+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:10:17.862164+0000","scrub_duration":0.002009186,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.4","version":"195'1125","reported_seq":6862,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:57.676169+0000","last_change":"2024-04-24T18:10:57.676133+0000","last_active":"2024-04-24T18:10:57.676169+0000","last_peered":"2024-04-24T18:10:57.676169+0000","last_clean":"2024-04-24T18:10:57.676169+0000","last_became_active":"2024-04-24T18:07:56.633494+0000","last_became_peered":"2024-04-24T18:07:56.633494+0000","last_unstale":"2024-04-24T18:10:57.676169+0000","last_undegraded":"2024-04-24T18:10:57.676169+0000","last_fullsized":"2024-04-24T18:10:57.676169+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1125","last_scrub_stamp":"2024-04-24T18:10:57.676048+0000","last_deep_scrub":"195'1125","last_deep_scrub_stamp":"2024-04-24T18:10:57.676048+0000","last_clean_scrub_stamp":"2024-04-24T18:10:57.676048+0000","objects_scrubbed":64,"log_size":1125,"log_dups_size":0,"ondisk_log_size":1125,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:13:51.782347+0000","scrub_duration":0.053100473000000002,"objects_trimmed":0,"snaptrim_duration":0.00082529200000000004,"stat_sum":{"num_bytes":0,"num_objects":64,"num_object_clones":0,"num_object_copies":128,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":64,"num_whiteouts":64,"num_read":9061,"num_read_kb":14482,"num_write":1281,"num_write_kb":16389,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":25,"num_bytes_recovered":155887,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2","version":"195'9","reported_seq":200,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.351053+0000","last_change":"2024-04-24T18:10:55.351010+0000","last_active":"2024-04-24T18:10:55.351053+0000","last_peered":"2024-04-24T18:10:55.351053+0000","last_clean":"2024-04-24T18:10:55.351053+0000","last_became_active":"2024-04-24T18:07:27.504830+0000","last_became_peered":"2024-04-24T18:07:27.504830+0000","last_unstale":"2024-04-24T18:10:55.351053+0000","last_undegraded":"2024-04-24T18:10:55.351053+0000","last_fullsized":"2024-04-24T18:10:55.351053+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'8","last_scrub_stamp":"2024-04-24T18:10:55.350928+0000","last_deep_scrub":"195'8","last_deep_scrub_stamp":"2024-04-24T18:10:55.350928+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.350928+0000","objects_scrubbed":4,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:47:58.630456+0000","scrub_duration":0.006846586,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":255,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":2,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":3,"num_bytes_hit_set_archive":255,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":2,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.923670+0000","last_change":"2024-04-24T18:10:55.923617+0000","last_active":"2024-04-24T18:10:55.923670+0000","last_peered":"2024-04-24T18:10:55.923670+0000","last_clean":"2024-04-24T18:10:55.923670+0000","last_became_active":"2024-04-24T18:10:18.331489+0000","last_became_peered":"2024-04-24T18:10:18.331489+0000","last_unstale":"2024-04-24T18:10:55.923670+0000","last_undegraded":"2024-04-24T18:10:55.923670+0000","last_fullsized":"2024-04-24T18:10:55.923670+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:55.923513+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:55.923513+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.923513+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:25:13.453258+0000","scrub_duration":0.001607608,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"195'1054","reported_seq":10365,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:00.128949+0000","last_change":"2024-04-24T18:11:00.128923+0000","last_active":"2024-04-24T18:11:00.128949+0000","last_peered":"2024-04-24T18:11:00.128949+0000","last_clean":"2024-04-24T18:11:00.128949+0000","last_became_active":"2024-04-24T18:08:56.897433+0000","last_became_peered":"2024-04-24T18:08:56.897433+0000","last_unstale":"2024-04-24T18:11:00.128949+0000","last_undegraded":"2024-04-24T18:11:00.128949+0000","last_fullsized":"2024-04-24T18:11:00.128949+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1054","last_scrub_stamp":"2024-04-24T18:11:00.128838+0000","last_deep_scrub":"195'1054","last_deep_scrub_stamp":"2024-04-24T18:11:00.128838+0000","last_clean_scrub_stamp":"2024-04-24T18:11:00.128838+0000","objects_scrubbed":73,"log_size":1054,"log_dups_size":0,"ondisk_log_size":1054,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:50:31.080209+0000","scrub_duration":0.056445503000000001,"objects_trimmed":0,"snaptrim_duration":0.00062859500000000002,"stat_sum":{"num_bytes":0,"num_objects":73,"num_object_clones":0,"num_object_copies":146,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":73,"num_whiteouts":73,"num_read":12086,"num_read_kb":44016,"num_write":1099,"num_write_kb":32724,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":116,"num_bytes_recovered":7548118,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":169,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:03.601186+0000","last_change":"2024-04-24T18:11:03.601136+0000","last_active":"2024-04-24T18:11:03.601186+0000","last_peered":"2024-04-24T18:11:03.601186+0000","last_clean":"2024-04-24T18:11:03.601186+0000","last_became_active":"2024-04-24T18:07:56.632586+0000","last_became_peered":"2024-04-24T18:07:56.632586+0000","last_unstale":"2024-04-24T18:11:03.601186+0000","last_undegraded":"2024-04-24T18:11:03.601186+0000","last_fullsized":"2024-04-24T18:11:03.601186+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:03.601020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:03.601020+0000","last_clean_scrub_stamp":"2024-04-24T18:11:03.601020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:36:54.636745+0000","scrub_duration":0.0017346939999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":212,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:00.947063+0000","last_change":"2024-04-24T18:11:00.947025+0000","last_active":"2024-04-24T18:11:00.947063+0000","last_peered":"2024-04-24T18:11:00.947063+0000","last_clean":"2024-04-24T18:11:00.947063+0000","last_became_active":"2024-04-24T18:10:18.331125+0000","last_became_peered":"2024-04-24T18:10:18.331125+0000","last_unstale":"2024-04-24T18:11:00.947063+0000","last_undegraded":"2024-04-24T18:11:00.947063+0000","last_fullsized":"2024-04-24T18:11:00.947063+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:00.946921+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:00.946921+0000","last_clean_scrub_stamp":"2024-04-24T18:11:00.946921+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:13:45.644151+0000","scrub_duration":0.0024498359999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.7","version":"195'618","reported_seq":5398,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.570019+0000","last_change":"2024-04-24T18:10:55.569997+0000","last_active":"2024-04-24T18:10:55.570019+0000","last_peered":"2024-04-24T18:10:55.570019+0000","last_clean":"2024-04-24T18:10:55.570019+0000","last_became_active":"2024-04-24T18:07:56.629925+0000","last_became_peered":"2024-04-24T18:07:56.629925+0000","last_unstale":"2024-04-24T18:10:55.570019+0000","last_undegraded":"2024-04-24T18:10:55.570019+0000","last_fullsized":"2024-04-24T18:10:55.570019+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'618","last_scrub_stamp":"2024-04-24T18:10:55.569917+0000","last_deep_scrub":"195'618","last_deep_scrub_stamp":"2024-04-24T18:10:55.569917+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.569917+0000","objects_scrubbed":44,"log_size":618,"log_dups_size":0,"ondisk_log_size":618,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T21:23:26.172108+0000","scrub_duration":0.024712272,"objects_trimmed":0,"snaptrim_duration":0.0010477189999999999,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6220,"num_read_kb":29443,"num_write":644,"num_write_kb":15073,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":3491812,"num_keys_recovered":11,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.1","version":"195'4","reported_seq":53,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:53.116565+0000","last_change":"2024-04-24T18:10:53.116543+0000","last_active":"2024-04-24T18:10:53.116565+0000","last_peered":"2024-04-24T18:10:53.116565+0000","last_clean":"2024-04-24T18:10:53.116565+0000","last_became_active":"2024-04-24T18:10:18.329333+0000","last_became_peered":"2024-04-24T18:10:18.329333+0000","last_unstale":"2024-04-24T18:10:53.116565+0000","last_undegraded":"2024-04-24T18:10:53.116565+0000","last_fullsized":"2024-04-24T18:10:53.116565+0000","mapping_epoch":169,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":170,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'4","last_scrub_stamp":"2024-04-24T18:10:53.116463+0000","last_deep_scrub":"195'4","last_deep_scrub_stamp":"2024-04-24T18:10:53.116463+0000","last_clean_scrub_stamp":"2024-04-24T18:10:53.116463+0000","objects_scrubbed":2,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:54:01.863529+0000","scrub_duration":0.00386317,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":83,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":1,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":1,"num_bytes_hit_set_archive":83,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":1,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"195'119","reported_seq":394,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:01.945228+0000","last_change":"2024-04-24T18:11:01.945199+0000","last_active":"2024-04-24T18:11:01.945228+0000","last_peered":"2024-04-24T18:11:01.945228+0000","last_clean":"2024-04-24T18:11:01.945228+0000","last_became_active":"2024-04-24T18:07:59.632517+0000","last_became_peered":"2024-04-24T18:07:59.632517+0000","last_unstale":"2024-04-24T18:11:01.945228+0000","last_undegraded":"2024-04-24T18:11:01.945228+0000","last_fullsized":"2024-04-24T18:11:01.945228+0000","mapping_epoch":48,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":49,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'119","last_scrub_stamp":"2024-04-24T18:11:01.945105+0000","last_deep_scrub":"195'119","last_deep_scrub_stamp":"2024-04-24T18:11:01.945105+0000","last_clean_scrub_stamp":"2024-04-24T18:11:01.945105+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:35:34.264274+0000","scrub_duration":0.0045221389999999997,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":184,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:08.646322+0000","last_change":"2024-04-24T18:11:08.646293+0000","last_active":"2024-04-24T18:11:08.646322+0000","last_peered":"2024-04-24T18:11:08.646322+0000","last_clean":"2024-04-24T18:11:08.646322+0000","last_became_active":"2024-04-24T18:08:53.875100+0000","last_became_peered":"2024-04-24T18:08:53.875100+0000","last_unstale":"2024-04-24T18:11:08.646322+0000","last_undegraded":"2024-04-24T18:11:08.646322+0000","last_fullsized":"2024-04-24T18:11:08.646322+0000","mapping_epoch":95,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":96,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:11:08.646187+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:11:08.646187+0000","last_clean_scrub_stamp":"2024-04-24T18:11:08.646187+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:24:14.590728+0000","scrub_duration":0.0021374290000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.6","version":"195'1135","reported_seq":10242,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:08.226615+0000","last_change":"2024-04-24T18:11:08.226586+0000","last_active":"2024-04-24T18:11:08.226615+0000","last_peered":"2024-04-24T18:11:08.226615+0000","last_clean":"2024-04-24T18:11:08.226615+0000","last_became_active":"2024-04-24T18:10:22.333339+0000","last_became_peered":"2024-04-24T18:10:22.333339+0000","last_unstale":"2024-04-24T18:11:08.226615+0000","last_undegraded":"2024-04-24T18:11:08.226615+0000","last_fullsized":"2024-04-24T18:11:08.226615+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1135","last_scrub_stamp":"2024-04-24T18:11:08.226450+0000","last_deep_scrub":"195'1135","last_deep_scrub_stamp":"2024-04-24T18:11:08.226450+0000","last_clean_scrub_stamp":"2024-04-24T18:11:08.226450+0000","objects_scrubbed":45,"log_size":1135,"log_dups_size":0,"ondisk_log_size":1135,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T05:18:04.488598+0000","scrub_duration":0.039250445000000002,"objects_trimmed":0,"snaptrim_duration":0.0012104310000000001,"stat_sum":{"num_bytes":0,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":45,"num_read":13278,"num_read_kb":32990,"num_write":1213,"num_write_kb":22164,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":204,"num_bytes_recovered":3060960,"num_keys_recovered":24,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"195'741","reported_seq":6249,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:13.974951+0000","last_change":"2024-04-24T18:11:13.974926+0000","last_active":"2024-04-24T18:11:13.974951+0000","last_peered":"2024-04-24T18:11:13.974951+0000","last_clean":"2024-04-24T18:11:13.974951+0000","last_became_active":"2024-04-24T18:07:36.548738+0000","last_became_peered":"2024-04-24T18:07:36.548738+0000","last_unstale":"2024-04-24T18:11:13.974951+0000","last_undegraded":"2024-04-24T18:11:13.974951+0000","last_fullsized":"2024-04-24T18:11:13.974951+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'741","last_scrub_stamp":"2024-04-24T18:11:13.974835+0000","last_deep_scrub":"195'741","last_deep_scrub_stamp":"2024-04-24T18:11:13.974835+0000","last_clean_scrub_stamp":"2024-04-24T18:11:13.974835+0000","objects_scrubbed":70,"log_size":741,"log_dups_size":0,"ondisk_log_size":741,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T18:47:41.121371+0000","scrub_duration":0.056156594999999997,"objects_trimmed":0,"snaptrim_duration":0.0015319419999999999,"stat_sum":{"num_bytes":0,"num_objects":70,"num_object_clones":0,"num_object_copies":140,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":70,"num_whiteouts":70,"num_read":6959,"num_read_kb":29956,"num_write":563,"num_write_kb":23523,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":194,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:54.390541+0000","last_change":"2024-04-24T18:10:54.390502+0000","last_active":"2024-04-24T18:10:54.390541+0000","last_peered":"2024-04-24T18:10:54.390541+0000","last_clean":"2024-04-24T18:10:54.390541+0000","last_became_active":"2024-04-24T18:07:20.404769+0000","last_became_peered":"2024-04-24T18:07:20.404769+0000","last_unstale":"2024-04-24T18:10:54.390541+0000","last_undegraded":"2024-04-24T18:10:54.390541+0000","last_fullsized":"2024-04-24T18:10:54.390541+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:54.390363+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:54.390363+0000","last_clean_scrub_stamp":"2024-04-24T18:10:54.390363+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T22:23:16.769713+0000","scrub_duration":0.0020000529999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"195'862","reported_seq":6205,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:59.339106+0000","last_change":"2024-04-24T18:10:59.339045+0000","last_active":"2024-04-24T18:10:59.339106+0000","last_peered":"2024-04-24T18:10:59.339106+0000","last_clean":"2024-04-24T18:10:59.339106+0000","last_became_active":"2024-04-24T18:07:36.550966+0000","last_became_peered":"2024-04-24T18:07:36.550966+0000","last_unstale":"2024-04-24T18:10:59.339106+0000","last_undegraded":"2024-04-24T18:10:59.339106+0000","last_fullsized":"2024-04-24T18:10:59.339106+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'862","last_scrub_stamp":"2024-04-24T18:10:59.338928+0000","last_deep_scrub":"195'862","last_deep_scrub_stamp":"2024-04-24T18:10:59.338928+0000","last_clean_scrub_stamp":"2024-04-24T18:10:59.338928+0000","objects_scrubbed":44,"log_size":862,"log_dups_size":0,"ondisk_log_size":862,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T20:28:38.073034+0000","scrub_duration":0.032138798000000003,"objects_trimmed":0,"snaptrim_duration":0.001114077,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":7385,"num_read_kb":18352,"num_write":839,"num_write_kb":9770,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":197,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:56.976176+0000","last_change":"2024-04-24T18:10:56.976138+0000","last_active":"2024-04-24T18:10:56.976176+0000","last_peered":"2024-04-24T18:10:56.976176+0000","last_clean":"2024-04-24T18:10:56.976176+0000","last_became_active":"2024-04-24T18:07:21.109281+0000","last_became_peered":"2024-04-24T18:07:21.109281+0000","last_unstale":"2024-04-24T18:10:56.976176+0000","last_undegraded":"2024-04-24T18:10:56.976176+0000","last_fullsized":"2024-04-24T18:10:56.976176+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-24T18:10:56.976030+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-24T18:10:56.976030+0000","last_clean_scrub_stamp":"2024-04-24T18:10:56.976030+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:52:11.277547+0000","scrub_duration":0.0015289189999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.8","version":"195'758","reported_seq":12227,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:01.753656+0000","last_change":"2024-04-24T18:11:01.753629+0000","last_active":"2024-04-24T18:11:01.753656+0000","last_peered":"2024-04-24T18:11:01.753656+0000","last_clean":"2024-04-24T18:11:01.753656+0000","last_became_active":"2024-04-24T18:07:36.545825+0000","last_became_peered":"2024-04-24T18:07:36.545825+0000","last_unstale":"2024-04-24T18:11:01.753656+0000","last_undegraded":"2024-04-24T18:11:01.753656+0000","last_fullsized":"2024-04-24T18:11:01.753656+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'758","last_scrub_stamp":"2024-04-24T18:11:01.753535+0000","last_deep_scrub":"195'758","last_deep_scrub_stamp":"2024-04-24T18:11:01.753535+0000","last_clean_scrub_stamp":"2024-04-24T18:11:01.753535+0000","objects_scrubbed":62,"log_size":758,"log_dups_size":0,"ondisk_log_size":758,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:56:29.522832+0000","scrub_duration":0.052474596999999998,"objects_trimmed":0,"snaptrim_duration":0.000718117,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":12680,"num_read_kb":39794,"num_write":654,"num_write_kb":18773,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"195'724","reported_seq":6377,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:25.942166+0000","last_change":"2024-04-24T18:11:25.942129+0000","last_active":"2024-04-24T18:11:25.942166+0000","last_peered":"2024-04-24T18:11:25.942166+0000","last_clean":"2024-04-24T18:11:25.942166+0000","last_became_active":"2024-04-24T18:07:56.629917+0000","last_became_peered":"2024-04-24T18:07:56.629917+0000","last_unstale":"2024-04-24T18:11:25.942166+0000","last_undegraded":"2024-04-24T18:11:25.942166+0000","last_fullsized":"2024-04-24T18:11:25.942166+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'724","last_scrub_stamp":"2024-04-24T18:11:25.942000+0000","last_deep_scrub":"195'724","last_deep_scrub_stamp":"2024-04-24T18:11:25.942000+0000","last_clean_scrub_stamp":"2024-04-24T18:11:25.942000+0000","objects_scrubbed":58,"log_size":724,"log_dups_size":0,"ondisk_log_size":724,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:17:09.552918+0000","scrub_duration":0.044509765999999999,"objects_trimmed":0,"snaptrim_duration":0.001170475,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":6125,"num_read_kb":18072,"num_write":672,"num_write_kb":12796,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":20,"num_bytes_recovered":1208069,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.a","version":"195'749","reported_seq":4003,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:57.145978+0000","last_change":"2024-04-24T18:10:57.145951+0000","last_active":"2024-04-24T18:10:57.145978+0000","last_peered":"2024-04-24T18:10:57.145978+0000","last_clean":"2024-04-24T18:10:57.145978+0000","last_became_active":"2024-04-24T18:08:56.895828+0000","last_became_peered":"2024-04-24T18:08:56.895828+0000","last_unstale":"2024-04-24T18:10:57.145978+0000","last_undegraded":"2024-04-24T18:10:57.145978+0000","last_fullsized":"2024-04-24T18:10:57.145978+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'749","last_scrub_stamp":"2024-04-24T18:10:57.145859+0000","last_deep_scrub":"195'749","last_deep_scrub_stamp":"2024-04-24T18:10:57.145859+0000","last_clean_scrub_stamp":"2024-04-24T18:10:57.145859+0000","objects_scrubbed":55,"log_size":749,"log_dups_size":0,"ondisk_log_size":749,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:45:28.736167+0000","scrub_duration":0.033221462,"objects_trimmed":0,"snaptrim_duration":0.00053847400000000005,"stat_sum":{"num_bytes":0,"num_objects":55,"num_object_clones":0,"num_object_copies":110,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":54,"num_read":3793,"num_read_kb":11584,"num_write":761,"num_write_kb":11416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":115,"num_bytes_recovered":2418909,"num_keys_recovered":40,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.b","version":"195'730","reported_seq":6373,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:55.178635+0000","last_change":"2024-04-24T18:10:55.178595+0000","last_active":"2024-04-24T18:10:55.178635+0000","last_peered":"2024-04-24T18:10:55.178635+0000","last_clean":"2024-04-24T18:10:55.178635+0000","last_became_active":"2024-04-24T18:10:21.328397+0000","last_became_peered":"2024-04-24T18:10:21.328397+0000","last_unstale":"2024-04-24T18:10:55.178635+0000","last_undegraded":"2024-04-24T18:10:55.178635+0000","last_fullsized":"2024-04-24T18:10:55.178635+0000","mapping_epoch":172,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":173,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'730","last_scrub_stamp":"2024-04-24T18:10:55.178457+0000","last_deep_scrub":"195'730","last_deep_scrub_stamp":"2024-04-24T18:10:55.178457+0000","last_clean_scrub_stamp":"2024-04-24T18:10:55.178457+0000","objects_scrubbed":48,"log_size":730,"log_dups_size":0,"ondisk_log_size":730,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:27:35.889998+0000","scrub_duration":0.033707577000000002,"objects_trimmed":0,"snaptrim_duration":0.0011667369999999999,"stat_sum":{"num_bytes":0,"num_objects":48,"num_object_clones":0,"num_object_copies":96,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":48,"num_whiteouts":48,"num_read":8575,"num_read_kb":17237,"num_write":686,"num_write_kb":13370,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":349,"num_bytes_recovered":61031,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"195'629","reported_seq":7133,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:06.927423+0000","last_change":"2024-04-24T18:11:06.927387+0000","last_active":"2024-04-24T18:11:06.927423+0000","last_peered":"2024-04-24T18:11:06.927423+0000","last_clean":"2024-04-24T18:11:06.927423+0000","last_became_active":"2024-04-24T18:07:36.551300+0000","last_became_peered":"2024-04-24T18:07:36.551300+0000","last_unstale":"2024-04-24T18:11:06.927423+0000","last_undegraded":"2024-04-24T18:11:06.927423+0000","last_fullsized":"2024-04-24T18:11:06.927423+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'629","last_scrub_stamp":"2024-04-24T18:11:06.927275+0000","last_deep_scrub":"195'629","last_deep_scrub_stamp":"2024-04-24T18:11:06.927275+0000","last_clean_scrub_stamp":"2024-04-24T18:11:06.927275+0000","objects_scrubbed":49,"log_size":629,"log_dups_size":0,"ondisk_log_size":629,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T04:33:44.035870+0000","scrub_duration":0.036822510000000003,"objects_trimmed":0,"snaptrim_duration":0.0013318570000000001,"stat_sum":{"num_bytes":0,"num_objects":49,"num_object_clones":0,"num_object_copies":98,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":49,"num_whiteouts":49,"num_read":7630,"num_read_kb":28742,"num_write":583,"num_write_kb":28670,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.d","version":"195'699","reported_seq":9358,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:15.967570+0000","last_change":"2024-04-24T18:11:15.967541+0000","last_active":"2024-04-24T18:11:15.967570+0000","last_peered":"2024-04-24T18:11:15.967570+0000","last_clean":"2024-04-24T18:11:15.967570+0000","last_became_active":"2024-04-24T18:07:36.548582+0000","last_became_peered":"2024-04-24T18:07:36.548582+0000","last_unstale":"2024-04-24T18:11:15.967570+0000","last_undegraded":"2024-04-24T18:11:15.967570+0000","last_fullsized":"2024-04-24T18:11:15.967570+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'699","last_scrub_stamp":"2024-04-24T18:11:15.967453+0000","last_deep_scrub":"195'699","last_deep_scrub_stamp":"2024-04-24T18:11:15.967453+0000","last_clean_scrub_stamp":"2024-04-24T18:11:15.967453+0000","objects_scrubbed":65,"log_size":699,"log_dups_size":0,"ondisk_log_size":699,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:27:49.001350+0000","scrub_duration":0.050498282999999998,"objects_trimmed":0,"snaptrim_duration":0.0030085200000000002,"stat_sum":{"num_bytes":0,"num_objects":65,"num_object_clones":0,"num_object_copies":130,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":65,"num_whiteouts":65,"num_read":8966,"num_read_kb":23944,"num_write":548,"num_write_kb":9568,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"195'640","reported_seq":7235,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:14.992707+0000","last_change":"2024-04-24T18:11:14.992678+0000","last_active":"2024-04-24T18:11:14.992707+0000","last_peered":"2024-04-24T18:11:14.992707+0000","last_clean":"2024-04-24T18:11:14.992707+0000","last_became_active":"2024-04-24T18:08:56.897303+0000","last_became_peered":"2024-04-24T18:08:56.897303+0000","last_unstale":"2024-04-24T18:11:14.992707+0000","last_undegraded":"2024-04-24T18:11:14.992707+0000","last_fullsized":"2024-04-24T18:11:14.992707+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'640","last_scrub_stamp":"2024-04-24T18:11:14.992574+0000","last_deep_scrub":"195'640","last_deep_scrub_stamp":"2024-04-24T18:11:14.992574+0000","last_clean_scrub_stamp":"2024-04-24T18:11:14.992574+0000","objects_scrubbed":44,"log_size":640,"log_dups_size":0,"ondisk_log_size":640,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:58:21.107187+0000","scrub_duration":0.034673908000000003,"objects_trimmed":0,"snaptrim_duration":0.0034553240000000001,"stat_sum":{"num_bytes":0,"num_objects":44,"num_object_clones":0,"num_object_copies":88,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":44,"num_whiteouts":44,"num_read":6837,"num_read_kb":16417,"num_write":572,"num_write_kb":13546,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":2206626,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"195'926","reported_seq":6502,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:10.688593+0000","last_change":"2024-04-24T18:11:10.688546+0000","last_active":"2024-04-24T18:11:10.688593+0000","last_peered":"2024-04-24T18:11:10.688593+0000","last_clean":"2024-04-24T18:11:10.688593+0000","last_became_active":"2024-04-24T18:07:56.628078+0000","last_became_peered":"2024-04-24T18:07:56.628078+0000","last_unstale":"2024-04-24T18:11:10.688593+0000","last_undegraded":"2024-04-24T18:11:10.688593+0000","last_fullsized":"2024-04-24T18:11:10.688593+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'926","last_scrub_stamp":"2024-04-24T18:11:10.688439+0000","last_deep_scrub":"195'926","last_deep_scrub_stamp":"2024-04-24T18:11:10.688439+0000","last_clean_scrub_stamp":"2024-04-24T18:11:10.688439+0000","objects_scrubbed":58,"log_size":926,"log_dups_size":0,"ondisk_log_size":926,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:01:55.416161+0000","scrub_duration":0.048479031999999998,"objects_trimmed":0,"snaptrim_duration":0.00095211600000000003,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8345,"num_read_kb":20880,"num_write":921,"num_write_kb":8630,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":21,"num_bytes_recovered":142614,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"195'721","reported_seq":7553,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:14.311182+0000","last_change":"2024-04-24T18:11:14.311146+0000","last_active":"2024-04-24T18:11:14.311182+0000","last_peered":"2024-04-24T18:11:14.311182+0000","last_clean":"2024-04-24T18:11:14.311182+0000","last_became_active":"2024-04-24T18:07:56.628257+0000","last_became_peered":"2024-04-24T18:07:56.628257+0000","last_unstale":"2024-04-24T18:11:14.311182+0000","last_undegraded":"2024-04-24T18:11:14.311182+0000","last_fullsized":"2024-04-24T18:11:14.311182+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'721","last_scrub_stamp":"2024-04-24T18:11:14.311047+0000","last_deep_scrub":"195'721","last_deep_scrub_stamp":"2024-04-24T18:11:14.311047+0000","last_clean_scrub_stamp":"2024-04-24T18:11:14.311047+0000","objects_scrubbed":58,"log_size":721,"log_dups_size":0,"ondisk_log_size":721,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T23:10:21.079191+0000","scrub_duration":0.047011873000000003,"objects_trimmed":0,"snaptrim_duration":0.0010801090000000001,"stat_sum":{"num_bytes":0,"num_objects":58,"num_object_clones":0,"num_object_copies":116,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":58,"num_whiteouts":58,"num_read":8160,"num_read_kb":18559,"num_write":661,"num_write_kb":14138,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":28,"num_bytes_recovered":636036,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.11","version":"195'827","reported_seq":9272,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:09.250017+0000","last_change":"2024-04-24T18:11:09.249966+0000","last_active":"2024-04-24T18:11:09.250017+0000","last_peered":"2024-04-24T18:11:09.250017+0000","last_clean":"2024-04-24T18:11:09.250017+0000","last_became_active":"2024-04-24T18:07:36.550563+0000","last_became_peered":"2024-04-24T18:07:36.550563+0000","last_unstale":"2024-04-24T18:11:09.250017+0000","last_undegraded":"2024-04-24T18:11:09.250017+0000","last_fullsized":"2024-04-24T18:11:09.250017+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'827","last_scrub_stamp":"2024-04-24T18:11:09.249858+0000","last_deep_scrub":"195'827","last_deep_scrub_stamp":"2024-04-24T18:11:09.249858+0000","last_clean_scrub_stamp":"2024-04-24T18:11:09.249858+0000","objects_scrubbed":62,"log_size":827,"log_dups_size":0,"ondisk_log_size":827,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:20:46.516321+0000","scrub_duration":0.049107976999999997,"objects_trimmed":0,"snaptrim_duration":0.001350265,"stat_sum":{"num_bytes":0,"num_objects":62,"num_object_clones":0,"num_object_copies":124,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":62,"num_whiteouts":62,"num_read":9568,"num_read_kb":48798,"num_write":865,"num_write_kb":30867,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"195'731","reported_seq":6604,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:56.616201+0000","last_change":"2024-04-24T18:10:56.616175+0000","last_active":"2024-04-24T18:10:56.616201+0000","last_peered":"2024-04-24T18:10:56.616201+0000","last_clean":"2024-04-24T18:10:56.616201+0000","last_became_active":"2024-04-24T18:08:56.895793+0000","last_became_peered":"2024-04-24T18:08:56.895793+0000","last_unstale":"2024-04-24T18:10:56.616201+0000","last_undegraded":"2024-04-24T18:10:56.616201+0000","last_fullsized":"2024-04-24T18:10:56.616201+0000","mapping_epoch":98,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":99,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'731","last_scrub_stamp":"2024-04-24T18:10:56.616086+0000","last_deep_scrub":"195'731","last_deep_scrub_stamp":"2024-04-24T18:10:56.616086+0000","last_clean_scrub_stamp":"2024-04-24T18:10:56.616086+0000","objects_scrubbed":69,"log_size":731,"log_dups_size":0,"ondisk_log_size":731,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T03:20:24.483369+0000","scrub_duration":0.038128871000000002,"objects_trimmed":0,"snaptrim_duration":0.00076871600000000002,"stat_sum":{"num_bytes":0,"num_objects":69,"num_object_clones":0,"num_object_copies":138,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":69,"num_whiteouts":69,"num_read":10377,"num_read_kb":31715,"num_write":586,"num_write_kb":8711,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":139,"num_bytes_recovered":5553458,"num_keys_recovered":20,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"195'1013","reported_seq":9207,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:10.924274+0000","last_change":"2024-04-24T18:11:10.924248+0000","last_active":"2024-04-24T18:11:10.924274+0000","last_peered":"2024-04-24T18:11:10.924274+0000","last_clean":"2024-04-24T18:11:10.924274+0000","last_became_active":"2024-04-24T18:08:57.902399+0000","last_became_peered":"2024-04-24T18:08:57.902399+0000","last_unstale":"2024-04-24T18:11:10.924274+0000","last_undegraded":"2024-04-24T18:11:10.924274+0000","last_fullsized":"2024-04-24T18:11:10.924274+0000","mapping_epoch":99,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":100,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1013","last_scrub_stamp":"2024-04-24T18:11:10.924160+0000","last_deep_scrub":"195'1013","last_deep_scrub_stamp":"2024-04-24T18:11:10.924160+0000","last_clean_scrub_stamp":"2024-04-24T18:11:10.924160+0000","objects_scrubbed":46,"log_size":1013,"log_dups_size":0,"ondisk_log_size":1013,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:23:51.640117+0000","scrub_duration":0.036631578999999997,"objects_trimmed":0,"snaptrim_duration":0.0032193669999999999,"stat_sum":{"num_bytes":0,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":46,"num_read":10393,"num_read_kb":22683,"num_write":1291,"num_write_kb":26660,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":102,"num_bytes_recovered":4079423,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.14","version":"195'729","reported_seq":8093,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:08.251570+0000","last_change":"2024-04-24T18:11:08.251543+0000","last_active":"2024-04-24T18:11:08.251570+0000","last_peered":"2024-04-24T18:11:08.251570+0000","last_clean":"2024-04-24T18:11:08.251570+0000","last_became_active":"2024-04-24T18:07:56.631920+0000","last_became_peered":"2024-04-24T18:07:56.631920+0000","last_unstale":"2024-04-24T18:11:08.251570+0000","last_undegraded":"2024-04-24T18:11:08.251570+0000","last_fullsized":"2024-04-24T18:11:08.251570+0000","mapping_epoch":45,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":46,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'729","last_scrub_stamp":"2024-04-24T18:11:08.251456+0000","last_deep_scrub":"195'729","last_deep_scrub_stamp":"2024-04-24T18:11:08.251456+0000","last_clean_scrub_stamp":"2024-04-24T18:11:08.251456+0000","objects_scrubbed":41,"log_size":729,"log_dups_size":0,"ondisk_log_size":729,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-25T19:35:01.655760+0000","scrub_duration":0.034614825000000002,"objects_trimmed":0,"snaptrim_duration":0.0014097140000000001,"stat_sum":{"num_bytes":0,"num_objects":41,"num_object_clones":0,"num_object_copies":82,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":41,"num_whiteouts":41,"num_read":9882,"num_read_kb":25724,"num_write":568,"num_write_kb":9852,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":58536,"num_keys_recovered":12,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"195'672","reported_seq":6108,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:11:02.759003+0000","last_change":"2024-04-24T18:11:02.758968+0000","last_active":"2024-04-24T18:11:02.759003+0000","last_peered":"2024-04-24T18:11:02.759003+0000","last_clean":"2024-04-24T18:11:02.759003+0000","last_became_active":"2024-04-24T18:07:36.545946+0000","last_became_peered":"2024-04-24T18:07:36.545946+0000","last_unstale":"2024-04-24T18:11:02.759003+0000","last_undegraded":"2024-04-24T18:11:02.759003+0000","last_fullsized":"2024-04-24T18:11:02.759003+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'672","last_scrub_stamp":"2024-04-24T18:11:02.758883+0000","last_deep_scrub":"195'672","last_deep_scrub_stamp":"2024-04-24T18:11:02.758883+0000","last_clean_scrub_stamp":"2024-04-24T18:11:02.758883+0000","objects_scrubbed":51,"log_size":672,"log_dups_size":0,"ondisk_log_size":672,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T02:44:10.975144+0000","scrub_duration":0.042076016000000001,"objects_trimmed":0,"snaptrim_duration":0.00065098100000000002,"stat_sum":{"num_bytes":0,"num_objects":51,"num_object_clones":0,"num_object_copies":102,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":51,"num_whiteouts":51,"num_read":7631,"num_read_kb":17555,"num_write":498,"num_write_kb":9152,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.16","version":"195'1226","reported_seq":7085,"reported_epoch":195,"state":"active+clean","last_fresh":"2024-04-24T18:10:58.193853+0000","last_change":"2024-04-24T18:10:58.193822+0000","last_active":"2024-04-24T18:10:58.193853+0000","last_peered":"2024-04-24T18:10:58.193853+0000","last_clean":"2024-04-24T18:10:58.193853+0000","last_became_active":"2024-04-24T18:10:22.333774+0000","last_became_peered":"2024-04-24T18:10:22.333774+0000","last_unstale":"2024-04-24T18:10:58.193853+0000","last_undegraded":"2024-04-24T18:10:58.193853+0000","last_fullsized":"2024-04-24T18:10:58.193853+0000","mapping_epoch":173,"log_start":"0'0","ondisk_log_start":"0'0","created":26,"last_epoch_clean":174,"parent":"0.0","parent_split_bits":0,"last_scrub":"195'1226","last_scrub_stamp":"2024-04-24T18:10:58.193720+0000","last_deep_scrub":"195'1226","last_deep_scrub_stamp":"2024-04-24T18:10:58.193720+0000","last_clean_scrub_stamp":"2024-04-24T18:10:58.193720+0000","objects_scrubbed":63,"log_size":1226,"log_dups_size":0,"ondisk_log_size":1226,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-26T00:40:15.346904+0000","scrub_duration":0.051315212999999998,"objects_trimmed":0,"snaptrim_duration":0.001325321,"stat_sum":{"num_bytes":0,"num_objects":63,"num_object_clones":0,"num_object_copies":126,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":63,"num_whiteouts":63,"num_read":8235,"num_read_kb":32695,"num_write":1497,"num_write_kb":16921,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":400,"num_bytes_recovered":61443,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":19,"num_objects":1792,"num_object_clones":0,"num_object_copies":3584,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1792,"num_whiteouts":1788,"num_read":292791,"num_read_kb":861574,"num_write":29102,"num_write_kb":531738,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2896,"num_bytes_recovered":45595312,"num_keys_recovered":161,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":38,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},"log_size":28056,"ondisk_log_size":28056,"up":64,"acting":64,"num_store_stats":8},{"poolid":3,"num_pg":4,"stat_sum":{"num_bytes":338,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":3,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":83,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":4,"num_bytes_hit_set_archive":338,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":3,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":13,"ondisk_log_size":13,"up":8,"acting":8,"num_store_stats":7},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":1771104,"data_compressed":28068,"data_compressed_allocated":110592,"data_compressed_original":1753088,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574903,"num_pgs":9,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307900,"kb_used_data":2356,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063940,"statfs":{"total":96636764160,"available":96321474560,"internally_reserved":0,"allocated":2412544,"data_stored":1791470,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.49299999999999999,"15min":0.47099999999999997},"min":{"1min":0.33800000000000002,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.69499999999999995,"5min":2.7799999999999998,"15min":2.7799999999999998},"last":0.752},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.49399999999999999,"15min":0.47599999999999998},"min":{"1min":0.35899999999999999,"5min":0.221,"15min":0.221},"max":{"1min":0.66400000000000003,"5min":2.585,"15min":2.585},"last":0.57999999999999996}]},{"osd":1,"last update":"Wed Apr 24 18:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56399999999999995,"15min":0.50700000000000001},"min":{"1min":0.42499999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70699999999999996,"5min":7.6740000000000004,"15min":7.6740000000000004},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.48799999999999999,"5min":0.47299999999999998,"15min":0.45900000000000002},"min":{"1min":0.34899999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.63900000000000001,"5min":2.008,"15min":2.008},"last":0.68400000000000005}]},{"osd":2,"last update":"Wed Apr 24 18:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.55400000000000005,"15min":0.51100000000000001},"min":{"1min":0.35099999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.63,"5min":7.3879999999999999,"15min":7.3879999999999999},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.54400000000000004,"15min":0.502},"min":{"1min":0.35499999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.67500000000000004,"5min":7.4749999999999996,"15min":7.4749999999999996},"last":0.64900000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.61499999999999999,"5min":0.55900000000000005,"15min":0.52300000000000002},"min":{"1min":0.308,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":2.1920000000000002,"5min":2.5499999999999998,"15min":2.5499999999999998},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.55000000000000004,"15min":0.53200000000000003},"min":{"1min":0.39000000000000001,"5min":0.313,"15min":0.313},"max":{"1min":2.3290000000000002,"5min":2.3290000000000002,"15min":2.3290000000000002},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.45700000000000002,"5min":0.45400000000000001,"15min":0.44400000000000001},"min":{"1min":0.33100000000000002,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":0.69699999999999995,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.435},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.496,"15min":0.501},"min":{"1min":0.374,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.68300000000000005,"5min":1.131,"15min":1.131},"last":0.46000000000000002}]},{"osd":5,"last update":"Wed Apr 24 18:11:21 2024","interfaces":[{"interface":"back","average":{"1min":0.51900000000000002,"5min":0.503,"15min":0.5},"min":{"1min":0.34999999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.69199999999999995,"5min":2.2109999999999999,"15min":2.2109999999999999},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.39100000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.753,"5min":2.6789999999999998,"15min":2.6789999999999998},"last":0.48199999999999998}]},{"osd":6,"last update":"Wed Apr 24 18:11:01 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.52900000000000003,"15min":0.52100000000000002},"min":{"1min":0.38800000000000001,"5min":0.34499999999999997,"15min":0.34499999999999997},"max":{"1min":0.72899999999999998,"5min":1.1579999999999999,"15min":1.1579999999999999},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.54600000000000004,"15min":0.53900000000000003},"min":{"1min":0.42999999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.75800000000000001,"5min":0.998,"15min":0.998},"last":0.499}]}]},{"osd":6,"up_from":95,"seq":408021893155,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":313220,"kb_used_data":2300,"kb_used_omap":31,"kb_used_meta":310880,"kb_avail":94058620,"statfs":{"total":96636764160,"available":96316026880,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":32737,"internal_metadata":318341151},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.52300000000000002,"5min":0.45100000000000001,"15min":0.439},"min":{"1min":0.33000000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.53600000000000003,"15min":0.53400000000000003},"min":{"1min":0.33500000000000002,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.89100000000000001,"5min":2.3260000000000001,"15min":2.3260000000000001},"last":0.625}]},{"osd":1,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.874,"15min":0.92600000000000005},"min":{"1min":0.41099999999999998,"5min":0.255,"15min":0.255},"max":{"1min":0.96099999999999997,"5min":10.071999999999999,"15min":10.071999999999999},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.54600000000000004,"15min":0.55000000000000004},"min":{"1min":0.35499999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.69099999999999995,"5min":1.26,"15min":1.26},"last":0.71199999999999997}]},{"osd":2,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57499999999999996,"15min":0.56699999999999995},"min":{"1min":0.35699999999999998,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":2.9609999999999999,"5min":2.9609999999999999,"15min":2.9609999999999999},"last":0.69799999999999995},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.61099999999999999,"15min":0.61699999999999999},"min":{"1min":0.42099999999999999,"5min":0.314,"15min":0.314},"max":{"1min":0.94899999999999995,"5min":2.306,"15min":2.306},"last":0.52000000000000002}]},{"osd":3,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.61199999999999999,"15min":0.61599999999999999},"min":{"1min":0.42599999999999999,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":0.78100000000000003,"5min":2.468,"15min":2.468},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.38300000000000001,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.79600000000000004,"5min":2.4300000000000002,"15min":2.4300000000000002},"last":0.55600000000000005}]},{"osd":4,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.42399999999999999,"15min":0.41499999999999998},"min":{"1min":0.32100000000000001,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.75,"5min":0.75,"15min":0.75},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.49399999999999999,"5min":0.48399999999999999,"15min":0.48199999999999998},"min":{"1min":0.307,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.79100000000000004,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.38700000000000001}]},{"osd":5,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.46600000000000003,"15min":0.46000000000000002},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.71999999999999997,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.373},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56899999999999995,"15min":0.57099999999999995},"min":{"1min":0.39400000000000002,"5min":0.34699999999999998,"15min":0.34699999999999998},"max":{"1min":0.81200000000000006,"5min":2.3599999999999999,"15min":2.3599999999999999},"last":0.41399999999999998}]},{"osd":7,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.59399999999999997,"15min":0.59999999999999998},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.81100000000000005,"5min":2.2669999999999999,"15min":2.2669999999999999},"last":0.435},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.51100000000000001,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.72599999999999998,"5min":0.72599999999999998,"15min":0.72599999999999998},"last":0.40500000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574902,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307884,"kb_used_data":2340,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063956,"statfs":{"total":96636764160,"available":96321490944,"internally_reserved":0,"allocated":2396160,"data_stored":1791470,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:11:23 2024","interfaces":[{"interface":"back","average":{"1min":0.35799999999999998,"5min":0.47499999999999998,"15min":0.59799999999999998},"min":{"1min":0.23999999999999999,"5min":0.19900000000000001,"15min":0.19900000000000001},"max":{"1min":0.56999999999999995,"5min":4.9249999999999998,"15min":4.9249999999999998},"last":0.34899999999999998},{"interface":"front","average":{"1min":0.40000000000000002,"5min":0.72599999999999998,"15min":1.071},"min":{"1min":0.28199999999999997,"5min":0.191,"15min":0.191},"max":{"1min":0.63400000000000001,"5min":14.15,"15min":14.15},"last":0.372}]},{"osd":2,"last update":"Wed Apr 24 18:11:23 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.45800000000000002,"15min":0.49199999999999999},"min":{"1min":0.27100000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.56000000000000005,"5min":1.306,"15min":1.306},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.79300000000000004,"15min":1.1319999999999999},"min":{"1min":0.25600000000000001,"5min":0.217,"15min":0.217},"max":{"1min":0.69099999999999995,"5min":14.118,"15min":14.118},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Apr 24 18:11:23 2024","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.76700000000000002,"15min":1.097},"min":{"1min":0.28699999999999998,"5min":0.248,"15min":0.248},"max":{"1min":0.57199999999999995,"5min":14.089,"15min":14.089},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.442,"5min":0.53500000000000003,"15min":0.65700000000000003},"min":{"1min":0.312,"5min":0.248,"15min":0.248},"max":{"1min":0.60399999999999998,"5min":5.1139999999999999,"15min":5.1139999999999999},"last":0.53800000000000003}]},{"osd":4,"last update":"Wed Apr 24 18:11:23 2024","interfaces":[{"interface":"back","average":{"1min":0.61599999999999999,"5min":0.57399999999999995,"15min":0.60999999999999999},"min":{"1min":0.373,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.997,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.91600000000000004,"15min":1.226},"min":{"1min":0.44700000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.0629999999999999,"5min":14.007999999999999,"15min":14.007999999999999},"last":0.56200000000000006}]},{"osd":5,"last update":"Wed Apr 24 18:11:23 2024","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.64600000000000002,"15min":0.749},"min":{"1min":0.36099999999999999,"5min":0.31900000000000001,"15min":0.31900000000000001},"max":{"1min":0.83099999999999996,"5min":5.0119999999999996,"15min":5.0119999999999996},"last":0.46700000000000003},{"interface":"front","average":{"1min":0.64000000000000001,"5min":0.624,"15min":0.66500000000000004},"min":{"1min":0.40500000000000003,"5min":0.32300000000000001,"15min":0.32300000000000001},"max":{"1min":0.85499999999999998,"5min":1.49,"15min":1.49},"last":0.57999999999999996}]},{"osd":6,"last update":"Wed Apr 24 18:10:58 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.55200000000000005,"15min":0.53900000000000003},"min":{"1min":0.436,"5min":0.379,"15min":0.379},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.60599999999999998,"5min":0.57599999999999996,"15min":0.57099999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":0.96799999999999997},"last":0.621}]},{"osd":7,"last update":"Wed Apr 24 18:11:23 2024","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.92600000000000005,"15min":1.244},"min":{"1min":0.41499999999999998,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.0269999999999999,"5min":14.051,"15min":14.051},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69399999999999995,"15min":0.80600000000000005},"min":{"1min":0.42299999999999999,"5min":0.373,"15min":0.373},"max":{"1min":0.879,"5min":5.0839999999999996,"15min":5.0839999999999996},"last":0.59999999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574903,"num_pgs":11,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311944,"kb_used_data":2304,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059896,"statfs":{"total":96636764160,"available":96317333504,"internally_reserved":0,"allocated":2359296,"data_stored":1201185,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":1.488,"5min":0.61199999999999999,"15min":0.48799999999999999},"min":{"1min":0.28599999999999998,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":18.981000000000002,"5min":18.981000000000002,"15min":18.981000000000002},"last":0.26500000000000001},{"interface":"front","average":{"1min":1.579,"5min":1.2050000000000001,"15min":1.621},"min":{"1min":0.30199999999999999,"5min":0.20200000000000001,"15min":0.20200000000000001},"max":{"1min":19.026,"5min":26.936,"15min":26.936},"last":0.40999999999999998}]},{"osd":2,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.753,"15min":1.0920000000000001},"min":{"1min":0.26500000000000001,"5min":0.23400000000000001,"15min":0.23400000000000001},"max":{"1min":1.0269999999999999,"5min":11.93,"15min":11.93},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.90400000000000003,"5min":0.54200000000000004,"15min":0.40000000000000002},"min":{"1min":0.34499999999999997,"5min":0.251,"15min":0.251},"max":{"1min":8.1639999999999997,"5min":8.1639999999999997,"15min":8.1639999999999997},"last":0.32500000000000001}]},{"osd":3,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.71099999999999997,"15min":0.95299999999999996},"min":{"1min":0.32000000000000001,"5min":0.222,"15min":0.222},"max":{"1min":1.095,"5min":14.371,"15min":14.371},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.498,"5min":0.76100000000000001,"15min":0.93899999999999995},"min":{"1min":0.317,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.70899999999999996,"5min":11.973000000000001,"15min":11.973000000000001},"last":0.38200000000000001}]},{"osd":4,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.81299999999999994,"15min":1.0760000000000001},"min":{"1min":0.44500000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.81799999999999995,"5min":14.414,"15min":14.414},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.626,"5min":0.57199999999999995,"15min":0.57799999999999996},"min":{"1min":0.46899999999999997,"5min":0.317,"15min":0.317},"max":{"1min":0.81699999999999995,"5min":1.329,"15min":1.329},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.69399999999999995,"15min":0.75600000000000001},"min":{"1min":0.41699999999999998,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.78800000000000003,"5min":6.5629999999999997,"15min":6.5629999999999997},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.85799999999999998,"15min":1.095},"min":{"1min":0.505,"5min":0.379,"15min":0.379},"max":{"1min":0.85599999999999998,"5min":14.452,"15min":14.452},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Apr 24 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.59799999999999998,"15min":0.59599999999999997},"min":{"1min":0.47499999999999998,"5min":0.38200000000000001,"15min":0.38200000000000001},"max":{"1min":0.88700000000000001,"5min":1.048,"15min":1.048},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.59399999999999997,"15min":0.58999999999999997},"min":{"1min":0.44600000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.65000000000000002}]},{"osd":7,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.68999999999999995,"5min":0.73999999999999999,"15min":0.79400000000000004},"min":{"1min":0.53800000000000003,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":1.0629999999999999,"5min":6.5300000000000002,"15min":6.5300000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.876,"15min":1.111},"min":{"1min":0.45900000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92200000000000004,"5min":14.486000000000001,"15min":14.486000000000001},"last":0.63100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574902,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303748,"kb_used_data":2300,"kb_used_omap":1,"kb_used_meta":301438,"kb_avail":94068092,"statfs":{"total":96636764160,"available":96325726208,"internally_reserved":0,"allocated":2355200,"data_stored":1201102,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":308672856},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.42099999999999999,"15min":0.41299999999999998},"min":{"1min":0.247,"5min":0.20399999999999999,"15min":0.20399999999999999},"max":{"1min":0.46899999999999997,"5min":4.109,"15min":4.109},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.39500000000000002,"5min":0.438,"15min":0.41999999999999998},"min":{"1min":0.253,"5min":0.20100000000000001,"15min":0.20100000000000001},"max":{"1min":0.58199999999999996,"5min":3.7410000000000001,"15min":3.7410000000000001},"last":0.50600000000000001}]},{"osd":1,"last update":"Wed Apr 24 18:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.376,"5min":0.43099999999999999,"15min":0.40600000000000003},"min":{"1min":0.24099999999999999,"5min":0.216,"15min":0.216},"max":{"1min":0.54300000000000004,"5min":4.8929999999999998,"15min":4.8929999999999998},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.40500000000000003,"5min":0.40899999999999997,"15min":0.39400000000000002},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.53000000000000003,"5min":1.619,"15min":1.619},"last":0.41999999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.40899999999999997,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.308,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.52900000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.42399999999999999,"5min":0.51500000000000001,"15min":0.48299999999999998},"min":{"1min":0.28799999999999998,"5min":0.247,"15min":0.247},"max":{"1min":0.57499999999999996,"5min":4.1399999999999997,"15min":4.1399999999999997},"last":0.433}]},{"osd":4,"last update":"Wed Apr 24 18:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.61699999999999999,"15min":0.57999999999999996},"min":{"1min":0.39000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.97499999999999998,"5min":5.3449999999999998,"15min":5.3449999999999998},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.56699999999999995,"15min":0.57199999999999995},"min":{"1min":0.443,"5min":0.35099999999999998,"15min":0.35099999999999998},"max":{"1min":0.91300000000000003,"5min":1.155,"15min":1.155},"last":0.58899999999999997}]},{"osd":5,"last update":"Wed Apr 24 18:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.61099999999999999,"15min":0.59899999999999998},"min":{"1min":0.46800000000000003,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.93300000000000005,"5min":2.9350000000000001,"15min":2.9350000000000001},"last":0.621},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.64700000000000002,"15min":0.623},"min":{"1min":0.46500000000000002,"5min":0.378,"15min":0.378},"max":{"1min":1.036,"5min":4.0679999999999996,"15min":4.0679999999999996},"last":0.81100000000000005}]},{"osd":6,"last update":"Wed Apr 24 18:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.99099999999999999,"15min":1.0469999999999999},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":1.006,"5min":10.587999999999999,"15min":10.587999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.63700000000000001,"5min":0.60199999999999998,"15min":0.59599999999999997},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.873,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.56399999999999995}]},{"osd":7,"last update":"Wed Apr 24 18:11:20 2024","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.73199999999999998,"15min":0.66000000000000003},"min":{"1min":0.503,"5min":0.376,"15min":0.376},"max":{"1min":0.91600000000000004,"5min":10.516,"15min":10.516},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.63800000000000001,"15min":0.623},"min":{"1min":0.42299999999999999,"5min":0.38,"15min":0.38},"max":{"1min":0.95799999999999996,"5min":2.9620000000000002,"15min":2.9620000000000002},"last":0.57599999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574903,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307860,"kb_used_data":2316,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063980,"statfs":{"total":96636764160,"available":96321515520,"internally_reserved":0,"allocated":2371584,"data_stored":1201376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51400000000000001,"5min":0.39600000000000002,"15min":0.36699999999999999},"min":{"1min":0.26400000000000001,"5min":0.20799999999999999,"15min":0.20799999999999999},"max":{"1min":2.2370000000000001,"5min":2.8119999999999998,"15min":2.8119999999999998},"last":0.29899999999999999},{"interface":"front","average":{"1min":1.131,"5min":0.501,"15min":0.38500000000000001},"min":{"1min":0.23100000000000001,"5min":0.23100000000000001,"15min":0.23100000000000001},"max":{"1min":15.452999999999999,"5min":15.452999999999999,"15min":15.452999999999999},"last":0.46100000000000002}]},{"osd":1,"last update":"Wed Apr 24 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":1.1080000000000001,"5min":0.53900000000000003,"15min":0.46700000000000003},"min":{"1min":0.29399999999999998,"5min":0.221,"15min":0.221},"max":{"1min":15.295,"5min":15.295,"15min":15.295},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.46400000000000002,"15min":0.42399999999999999},"min":{"1min":0.27500000000000002,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":4.1779999999999999,"5min":4.1779999999999999,"15min":4.1779999999999999},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Apr 24 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60799999999999998,"5min":0.42899999999999999,"15min":0.39300000000000002},"min":{"1min":0.29399999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":4.0899999999999999,"5min":4.0899999999999999,"15min":4.0899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":1.121,"5min":0.60599999999999998,"15min":0.49399999999999999},"min":{"1min":0.315,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":11.977,"5min":11.977,"15min":11.977},"last":0.41099999999999998}]},{"osd":4,"last update":"Wed Apr 24 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.57599999999999996,"15min":0.56399999999999995},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.90500000000000003,"5min":3.2360000000000002,"15min":3.2360000000000002},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.54200000000000004,"15min":0.53900000000000003},"min":{"1min":0.40300000000000002,"5min":0.314,"15min":0.314},"max":{"1min":0.91400000000000003,"5min":1.3129999999999999,"15min":1.3129999999999999},"last":0.47699999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.61199999999999999,"15min":0.58999999999999997},"min":{"1min":0.45800000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":2.323,"5min":3.2130000000000001,"15min":3.2130000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.61399999999999999,"5min":0.59899999999999998,"15min":0.58899999999999997},"min":{"1min":0.39900000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.96499999999999997,"5min":3.4049999999999998,"15min":3.4049999999999998},"last":0.498}]},{"osd":6,"last update":"Wed Apr 24 18:10:55 2024","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":2.282,"5min":2.7389999999999999,"15min":2.7389999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.61699999999999999,"5min":0.66700000000000004,"15min":0.67500000000000004},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.93700000000000006,"5min":2.7669999999999999,"15min":2.7669999999999999},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed Apr 24 18:11:28 2024","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.66300000000000003,"15min":0.60799999999999998},"min":{"1min":0.496,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":2.3559999999999999,"5min":3.8180000000000001,"15min":3.8180000000000001},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.78500000000000003,"15min":0.66800000000000004},"min":{"1min":0.53200000000000003,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":2.3849999999999998,"5min":15.4,"15min":15.4},"last":0.621}]}]},{"osd":4,"up_from":13,"seq":55834574904,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":311948,"kb_used_data":2308,"kb_used_omap":1,"kb_used_meta":309630,"kb_avail":94059892,"statfs":{"total":96636764160,"available":96317329408,"internally_reserved":0,"allocated":2363392,"data_stored":1201204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":317061464},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:11:27 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.48899999999999999,"15min":0.48199999999999998},"min":{"1min":0.33800000000000002,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.76500000000000001,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.52200000000000002,"15min":0.50600000000000001},"min":{"1min":0.42999999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.80800000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.58799999999999997}]},{"osd":1,"last update":"Wed Apr 24 18:11:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.48699999999999999,"15min":0.498},"min":{"1min":0.373,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":0.66200000000000003,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.436},{"interface":"front","average":{"1min":0.53900000000000003,"5min":0.45800000000000002,"15min":0.45600000000000002},"min":{"1min":0.371,"5min":0.251,"15min":0.251},"max":{"1min":0.753,"5min":0.753,"15min":0.753},"last":0.53800000000000003}]},{"osd":2,"last update":"Wed Apr 24 18:11:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54800000000000004,"5min":0.47099999999999997,"15min":0.46200000000000002},"min":{"1min":0.39800000000000002,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.80100000000000005,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.59599999999999997,"5min":0.505,"15min":0.501},"min":{"1min":0.35999999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.79000000000000004,"5min":1.153,"15min":1.153},"last":0.55400000000000005}]},{"osd":3,"last update":"Wed Apr 24 18:11:27 2024","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.498,"15min":0.49399999999999999},"min":{"1min":0.435,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.73699999999999999,"5min":0.80600000000000005,"15min":0.80600000000000005},"last":0.501},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.53200000000000003,"15min":0.50800000000000001},"min":{"1min":0.39200000000000002,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.71699999999999997,"5min":2.5019999999999998,"15min":2.5019999999999998},"last":0.60699999999999998}]},{"osd":5,"last update":"Wed Apr 24 18:11:27 2024","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.35899999999999999,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.65700000000000003,"5min":1.373,"15min":1.373},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.52100000000000002,"5min":0.44900000000000001,"15min":0.45200000000000001},"min":{"1min":0.315,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.76100000000000001,"5min":0.93400000000000005,"15min":0.93400000000000005},"last":0.48999999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:10:57 2024","interfaces":[{"interface":"back","average":{"1min":0.46300000000000002,"5min":0.41399999999999998,"15min":0.40600000000000003},"min":{"1min":0.29699999999999999,"5min":0.248,"15min":0.248},"max":{"1min":0.70999999999999996,"5min":0.70999999999999996,"15min":0.70999999999999996},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.48599999999999999,"5min":0.46899999999999997,"15min":0.46600000000000003},"min":{"1min":0.318,"5min":0.315,"15min":0.315},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.627}]},{"osd":7,"last update":"Wed Apr 24 18:11:27 2024","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.496,"15min":0.50900000000000001},"min":{"1min":0.33000000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.77600000000000002,"5min":0.88400000000000001,"15min":0.88400000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.53300000000000003,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.25600000000000001,"15min":0.25600000000000001},"max":{"1min":0.79900000000000004,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.56899999999999995}]}]},{"osd":5,"up_from":13,"seq":55834574903,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":307856,"kb_used_data":2312,"kb_used_omap":1,"kb_used_meta":305534,"kb_avail":94063984,"statfs":{"total":96636764160,"available":96321519616,"internally_reserved":0,"allocated":2367488,"data_stored":1201357,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":312867160},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.47599999999999998,"15min":0.44800000000000001},"min":{"1min":0.40000000000000002,"5min":0.189,"15min":0.189},"max":{"1min":3.5840000000000001,"5min":3.5840000000000001,"15min":3.5840000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.47099999999999997},"min":{"1min":0.38,"5min":0.222,"15min":0.222},"max":{"1min":0.73699999999999999,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.70599999999999996}]},{"osd":1,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.47799999999999998,"15min":0.45400000000000001},"min":{"1min":0.39500000000000002,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.53500000000000003,"15min":0.51100000000000001},"min":{"1min":0.39900000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":3.7130000000000001,"5min":3.7130000000000001,"15min":3.7130000000000001},"last":0.55200000000000005}]},{"osd":2,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.58399999999999996,"5min":0.51700000000000002,"15min":0.54100000000000004},"min":{"1min":0.43099999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.407,"5min":1.407,"15min":1.407},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.61899999999999999,"5min":0.51900000000000002,"15min":0.53600000000000003},"min":{"1min":0.42199999999999999,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":1.361,"5min":1.377,"15min":1.377},"last":0.72899999999999998}]},{"osd":3,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.59399999999999997,"5min":0.57699999999999996,"15min":0.63100000000000001},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.98299999999999998,"5min":1.899,"15min":1.899},"last":0.63},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.57799999999999996,"15min":0.63300000000000001},"min":{"1min":0.35499999999999998,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":0.997,"5min":1.881,"15min":1.881},"last":0.67000000000000004}]},{"osd":4,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.46000000000000002,"15min":0.5},"min":{"1min":0.32700000000000001,"5min":0.25,"15min":0.25},"max":{"1min":0.72799999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.503,"15min":0.55400000000000005},"min":{"1min":0.317,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.748,"5min":1.0640000000000001,"15min":1.0640000000000001},"last":0.42399999999999999}]},{"osd":6,"last update":"Wed Apr 24 18:10:56 2024","interfaces":[{"interface":"back","average":{"1min":0.496,"5min":0.41199999999999998,"15min":0.39800000000000002},"min":{"1min":0.312,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.43099999999999999,"15min":0.42299999999999999},"min":{"1min":0.34599999999999997,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.754,"5min":0.754,"15min":0.754},"last":0.44800000000000001}]},{"osd":7,"last update":"Wed Apr 24 18:11:26 2024","interfaces":[{"interface":"back","average":{"1min":0.56999999999999995,"5min":0.53600000000000003,"15min":0.57599999999999996},"min":{"1min":0.39700000000000002,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.83599999999999997,"5min":1.016,"15min":1.016},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.54100000000000004,"15min":0.58099999999999996},"min":{"1min":0.44800000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.85599999999999998,"5min":1.1639999999999999,"15min":1.1639999999999999},"last":0.53800000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":590368,"data_compressed":9702,"data_compressed_allocated":36864,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":590368,"data_compressed":8664,"data_compressed_allocated":36864,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":83,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":9738,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-24T18:11:36.956 DEBUG:teuthology.orchestra.run.smithi049:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-24T18:11:37.306 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-24T18:11:37.306 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-24T18:11:37.306 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-24T18:11:37.306 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.344 INFO:tasks.ceph.osd.0:Stopped 2024-04-24T18:11:37.344 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-24T18:11:37.344 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.415 INFO:tasks.ceph.osd.1:Stopped 2024-04-24T18:11:37.415 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-24T18:11:37.415 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.487 INFO:tasks.ceph.osd.2:Stopped 2024-04-24T18:11:37.487 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-24T18:11:37.487 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.558 INFO:tasks.ceph.osd.3:Stopped 2024-04-24T18:11:37.559 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-04-24T18:11:37.559 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.600 INFO:tasks.ceph.osd.4:Stopped 2024-04-24T18:11:37.600 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-04-24T18:11:37.600 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.638 INFO:tasks.ceph.osd.5:Stopped 2024-04-24T18:11:37.638 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-04-24T18:11:37.638 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.708 INFO:tasks.ceph.osd.6:Stopped 2024-04-24T18:11:37.708 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-04-24T18:11:37.709 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.779 INFO:tasks.ceph.osd.7:Stopped 2024-04-24T18:11:37.779 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-24T18:11:37.779 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-04-24T18:11:37.779 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.801 INFO:tasks.ceph.mgr.y:Stopped 2024-04-24T18:11:37.802 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-24T18:11:37.802 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.845 INFO:tasks.ceph.mgr.x:Stopped 2024-04-24T18:11:37.845 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-24T18:11:37.846 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-24T18:11:37.846 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.856 INFO:tasks.ceph.mon.a:Stopped 2024-04-24T18:11:37.856 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-04-24T18:11:37.856 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.906 INFO:tasks.ceph.mon.c:Stopped 2024-04-24T18:11:37.907 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-04-24T18:11:37.907 INFO:teuthology.orchestra.run:waiting for 300 2024-04-24T18:11:37.918 INFO:tasks.ceph.mon.b:Stopped 2024-04-24T18:11:37.918 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-24T18:11:37.919 DEBUG:teuthology.orchestra.run.smithi049:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | head -n 1 2024-04-24T18:11:37.981 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:11:37.981 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-24T18:11:38.258 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:11:38.259 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-24T18:11:38.354 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:11:38.354 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-24T18:11:38.420 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:11:38.420 DEBUG:teuthology.orchestra.run.smithi049:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-24T18:11:38.487 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:11:38.487 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-04-24T18:11:38.671 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:11:38.671 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-04-24T18:11:38.741 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:11:38.741 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-04-24T18:11:38.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:11:38.799 DEBUG:teuthology.orchestra.run.smithi078:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-04-24T18:11:38.888 INFO:tasks.ceph:Archiving mon data... 2024-04-24T18:11:38.889 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/data/mon.a.tgz 2024-04-24T18:11:38.890 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-04-24T18:11:38.908 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.9tncLbCFbX 2024-04-24T18:11:38.909 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.9tncLbCFbX 2024-04-24T18:11:39.075 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.9tncLbCFbX 2024-04-24T18:11:39.162 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.9tncLbCFbX is 423KB 2024-04-24T18:11:39.259 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.9tncLbCFbX 2024-04-24T18:11:39.276 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/data/mon.c.tgz 2024-04-24T18:11:39.276 DEBUG:teuthology.orchestra.run.smithi049:> mktemp 2024-04-24T18:11:39.333 INFO:teuthology.orchestra.run.smithi049.stdout:/tmp/tmp.NPdKAgQ11q 2024-04-24T18:11:39.333 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.NPdKAgQ11q 2024-04-24T18:11:39.524 DEBUG:teuthology.orchestra.run.smithi049:> sudo chmod 0666 /tmp/tmp.NPdKAgQ11q 2024-04-24T18:11:39.611 DEBUG:teuthology.orchestra.remote:smithi049:/tmp/tmp.NPdKAgQ11q is 587KB 2024-04-24T18:11:39.681 DEBUG:teuthology.orchestra.run.smithi049:> rm -fr /tmp/tmp.NPdKAgQ11q 2024-04-24T18:11:39.698 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/data/mon.b.tgz 2024-04-24T18:11:39.698 DEBUG:teuthology.orchestra.run.smithi078:> mktemp 2024-04-24T18:11:39.718 INFO:teuthology.orchestra.run.smithi078.stdout:/tmp/tmp.Jku4OVlYaV 2024-04-24T18:11:39.718 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.Jku4OVlYaV 2024-04-24T18:11:39.924 DEBUG:teuthology.orchestra.run.smithi078:> sudo chmod 0666 /tmp/tmp.Jku4OVlYaV 2024-04-24T18:11:40.009 DEBUG:teuthology.orchestra.remote:smithi078:/tmp/tmp.Jku4OVlYaV is 587KB 2024-04-24T18:11:40.108 DEBUG:teuthology.orchestra.run.smithi078:> rm -fr /tmp/tmp.Jku4OVlYaV 2024-04-24T18:11:40.124 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-24T18:11:40.124 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T18:11:40.126 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-24T18:11:40.190 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2024-04-24T18:11:40.190 DEBUG:teuthology.orchestra.run.smithi049:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-24T18:11:40.195 DEBUG:teuthology.orchestra.run.smithi078:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2024-04-24T18:11:40.222 INFO:teuthology.orchestra.run.smithi049.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-24T18:11:40.257 INFO:teuthology.orchestra.run.smithi078.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2024-04-24T18:11:40.263 INFO:tasks.ceph:Archiving crash dumps... 2024-04-24T18:11:40.264 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/remote/smithi049/crash 2024-04-24T18:11:40.265 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T18:11:40.297 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/remote/smithi078/crash 2024-04-24T18:11:40.298 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-24T18:11:40.329 INFO:tasks.ceph:Compressing logs... 2024-04-24T18:11:40.329 DEBUG:teuthology.orchestra.run.smithi049:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T18:11:40.341 DEBUG:teuthology.orchestra.run.smithi078:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-24T18:11:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65632.log 2024-04-24T18:11:40.366 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47678.log 2024-04-24T18:11:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71559.log 2024-04-24T18:11:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65632.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65632.log.gz 2024-04-24T18:11:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47678.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61732.log 2024-04-24T18:11:40.396 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47678.log.gz 2024-04-24T18:11:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74463.log 2024-04-24T18:11:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53574.log 2024-04-24T18:11:40.396 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54431.log 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71559.log: /var/log/ceph/ceph-client.admin.61732.log: /var/log/ceph/ceph-client.admin.74463.log: gzip -5 --verbose /var/log/ceph/ceph-client.admin.53574.log: -- /var/log/ceph/ceph-client.admin.72264.log 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61732.log.gz -- replaced with /var/log/ceph/ceph-client.admin.71559.log.gz 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74463.log.gz 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53574.log.gz 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54431.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48248.log 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54431.log.gz 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58828.log 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72264.log.gz 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48248.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51792.log 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48248.log.gz 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58828.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58828.log.gz 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51792.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61232.log 2024-04-24T18:11:40.397 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51792.log.gz 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50167.log 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61232.log: gzip -5 0.0%--verbose -- replaced with /var/log/ceph/ceph-client.admin.61232.log.gz-- 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60451.log 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50167.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49618.log 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50167.log.gz 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66686.log 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60451.log.gz 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49618.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49040.log 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49618.log.gz 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66686.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.73905.log -- replaced with /var/log/ceph/ceph-client.admin.66686.log.gz 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74419.log 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49040.log.gz 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73905.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54900.log 2024-04-24T18:11:40.398 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73905.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74419.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72270.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74419.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54900.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54900.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71789.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72270.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74089.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72270.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49318.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71789.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74089.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71329.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74089.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49318.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49318.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72265.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71329.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48894.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71329.log.gz 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72265.log: gzip -5 --verbose 0.0%-- -- replaced with /var/log/ceph/ceph-client.admin.72265.log.gz/var/log/ceph/ceph-client.admin.47558.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47510.log 2024-04-24T18:11:40.399 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48894.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73010.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47558.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72268.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47510.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73010.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73010.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48994.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73261.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72268.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72268.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54002.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48994.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51928.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73261.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54002.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.54002.log.gz--verbose 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:-- /var/log/ceph/ceph-client.admin.50945.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68104.log 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51928.log.gz 2024-04-24T18:11:40.400 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50945.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58328.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50945.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68104.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50877.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68104.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72584.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58328.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60497.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50877.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50877.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49548.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72584.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60497.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74043.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60497.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66294.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49548.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49548.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52014.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74043.log.gz 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69044.log 2024-04-24T18:11:40.401 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66294.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59811.log 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52014.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52014.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53026.log 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69044.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57118.log 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59811.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47951.log 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53026.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67473.log 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57118.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63360.log 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47951.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67473.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.67473.log.gz 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49272.log 2024-04-24T18:11:40.402 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63360.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72990.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63360.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49272.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49272.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65910.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55768.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72990.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72990.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65910.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.48177.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.65910.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61646.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55768.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48177.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69517.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48177.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56230.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61646.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61646.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69517.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58531.log 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69517.log.gz 2024-04-24T18:11:40.403 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72263.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56230.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58531.log: gzip -5 --verbose 0.0%-- /var/log/ceph/ceph-client.admin.63655.log -- replaced with /var/log/ceph/ceph-client.admin.58531.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72633.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72263.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72263.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63655.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72597.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63655.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72633.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53118.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60352.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72597.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53118.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49777.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53118.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60352.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47940.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60352.log.gz 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65013.log 2024-04-24T18:11:40.404 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49777.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49777.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47940.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52178.log 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47940.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65013.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48299.log 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65013.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52178.log.gzgzip 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.72019.log 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58700.log 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48299.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48299.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70584.log 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72019.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58700.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63433.log 0.0% 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.58700.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70584.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63433.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47944.log 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63433.log.gz 2024-04-24T18:11:40.405 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60544.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47944.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.47944.log.gz/var/log/ceph/ceph-client.admin.66412.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60544.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63928.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60544.log.gz 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66412.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50373.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66412.log.gz 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59173.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63928.log.gz 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50373.log.gz 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47939.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59173.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67922.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59173.log.gz 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69665.log 0.0% 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.47939.log.gz 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67922.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48712.log 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67922.log.gz 2024-04-24T18:11:40.406 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69665.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.61419.log 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69665.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48712.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62782.log 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48712.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61419.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61419.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73859.log 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62782.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72870.log 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62782.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73859.log: gzip -5 --verbose -- 0.0%/var/log/ceph/ceph-client.admin.48940.log -- replaced with /var/log/ceph/ceph-client.admin.73859.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72870.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71186.log 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72870.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48940.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.48940.log.gz-5 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:--verbose -- /var/log/ceph/ceph-client.admin.62989.log 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71835.log 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71186.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71186.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62989.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62989.log.gz 2024-04-24T18:11:40.407 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64374.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73721.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71835.log.gz 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.64374.log: gzip -5 0.0%--verbose -- -- replaced with /var/log/ceph/ceph-client.admin.64374.log.gz/var/log/ceph/ceph-client.admin.73445.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73721.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61777.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73721.log.gz 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73445.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.73445.log.gz 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:--verbose -- /var/log/ceph/ceph-client.admin.74319.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61777.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47948.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61777.log.gz 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74319.log: gzip -5 0.0%--verbose -- replaced with /var/log/ceph/ceph-client.admin.74319.log.gz-- 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62460.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47948.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47948.log.gz 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62460.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.62460.log.gz--verbose 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:-- /var/log/ceph/ceph-client.admin.72993.log 2024-04-24T18:11:40.408 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73537.log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.5.log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool..log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.50537.log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/objectstore_tool..log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53264.log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/objectstore_tool.50537.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43194.log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/objectstore_tool.53264.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43337.log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43194.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43194.log.gz 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43337.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43337.log.gz 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43051.log 2024-04-24T18:11:40.417 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43051.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-osd.7.log: 95.2% -- replaced with /var/log/ceph/objectstore_tool.50537.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr: 95.1% -- replaced with /var/log/ceph/objectstore_tool.53264.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43385.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43385.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43385.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-osd.6.log: 83.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-mon.b.log: 88.2% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43247.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43148.log 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43247.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43247.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43148.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43148.log.gz 2024-04-24T18:11:40.418 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43293.log 2024-04-24T18:11:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52354.log 2024-04-24T18:11:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43293.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43293.log.gz 2024-04-24T18:11:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/objectstore_tool.52354.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-24T18:11:40.419 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-04-24T18:11:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-osd.4.log: 95.6% -- replaced with /var/log/ceph/objectstore_tool.52354.log.gz 2024-04-24T18:11:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.51518.log 2024-04-24T18:11:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/objectstore_tool.51518.log: 95.4% -- replaced with /var/log/ceph/objectstore_tool..log.gz 2024-04-24T18:11:40.428 INFO:teuthology.orchestra.run.smithi078.stderr: 95.2% -- replaced with /var/log/ceph/objectstore_tool.51518.log.gz 2024-04-24T18:11:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43098.log 2024-04-24T18:11:40.428 INFO:teuthology.orchestra.run.smithi078.stderr:/var/log/ceph/ceph-client.admin.43098.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43098.log.gz 2024-04-24T18:11:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72993.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70896.log 2024-04-24T18:11:40.430 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72993.log.gz 2024-04-24T18:11:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73537.log: gzip 0.0%-5 -- replaced with /var/log/ceph/ceph-client.admin.73537.log.gz--verbose 2024-04-24T18:11:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:-- /var/log/ceph/ceph-client.admin.68697.log 2024-04-24T18:11:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62302.log 2024-04-24T18:11:40.430 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70896.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70896.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68697.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64557.log 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68697.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62302.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65515.log 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62302.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr: 91.0% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64210.log 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.64557.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64557.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65515.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65059.log 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65515.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.64210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58921.log 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64210.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65059.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.65059.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:-5 --verbose -- /var/log/ceph/ceph-client.admin.60520.log 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68499.log 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58921.log.gz 2024-04-24T18:11:40.431 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60520.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60520.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59741.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68499.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log: /var/log/ceph/ceph-client.admin.71973.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59741.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62675.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59741.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71973.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52318.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71973.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61314.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62675.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52318.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73399.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52318.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61314.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61314.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73399.log.gz 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68554.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51446.log 2024-04-24T18:11:40.432 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68554.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68839.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51446.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51446.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68839.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68839.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64046.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73307.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.64046.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64046.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73307.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73307.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59373.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72869.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59373.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72869.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72869.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59232.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51815.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59232.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72988.log 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59232.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51815.log.gz 2024-04-24T18:11:40.433 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72988.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72988.log.gz 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67826.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60837.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48642.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67826.log.gz 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60837.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60837.log.gz 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48642.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48642.log.gz 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66938.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71232.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66938.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59517.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66938.log.gz 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71232.log.gz 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59517.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59517.log.gz 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70097.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49088.log 2024-04-24T18:11:40.434 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70097.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49088.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49088.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43472.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48212.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43472.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43472.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48212.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48212.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48696.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67665.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67370.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48696.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48696.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67665.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67665.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67370.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67370.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71743.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69139.log 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71743.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71743.log.gz 2024-04-24T18:11:40.435 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69139.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57332.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69139.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72272.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57332.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57332.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72272.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72983.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72272.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70668.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72983.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70668.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70668.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54260.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48638.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54260.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48638.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37092.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51152.log 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.tmp-client.admin.37092.log: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37092.log.gz 2024-04-24T18:11:40.436 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51152.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51152.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49502.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58257.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49502.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49502.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58257.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58257.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48250.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52570.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48250.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48250.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52570.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49180.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66137.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49180.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49180.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66137.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66137.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53323.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51640.log 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53323.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53323.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51640.log.gz 2024-04-24T18:11:40.437 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52179.log 2024-04-24T18:11:40.438 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63883.log 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52179.log.gz 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63883.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63883.log.gz 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60715.log 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55935.log 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60715.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60715.log.gz 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55935.log.gz 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54104.log 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63290.log 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54104.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54104.log.gz 2024-04-24T18:11:40.442 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63290.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63290.log.gz 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64601.log 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47447.log 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.64601.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64601.log.gz 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47447.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47447.log.gz 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71697.log 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr: 93.3% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71697.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71697.log.gz 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71513.log 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68584.log 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71513.log.gz 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68584.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68584.log.gz 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43126.log 2024-04-24T18:11:40.443 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70418.log 2024-04-24T18:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43126.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43126.log.gz 2024-04-24T18:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70418.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70418.log.gz 2024-04-24T18:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47728.log 2024-04-24T18:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72985.log 2024-04-24T18:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47728.log.gz 2024-04-24T18:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72985.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72985.log.gz 2024-04-24T18:11:40.444 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52451.log 2024-04-24T18:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51303.log 2024-04-24T18:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52451.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52451.log.gz 2024-04-24T18:11:40.446 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51303.log.gz 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47938.log 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62182.log 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47938.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47938.log.gz 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62182.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62182.log.gz 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61003.log 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72640.log 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61003.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61003.log.gz 2024-04-24T18:11:40.449 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72640.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72640.log.gz 2024-04-24T18:11:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74135.log 2024-04-24T18:11:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49673.log 2024-04-24T18:11:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74135.log.gz 2024-04-24T18:11:40.450 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.0.49673.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55455.log 2024-04-24T18:11:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55455.log.gz 2024-04-24T18:11:40.452 INFO:teuthology.orchestra.run.smithi049.stderr: 95.6% -- replaced with /var/log/ceph/ceph-client.0.49673.log.gz 2024-04-24T18:11:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51884.log 2024-04-24T18:11:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71605.log 2024-04-24T18:11:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51884.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51884.log.gz 2024-04-24T18:11:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71605.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71605.log.gz 2024-04-24T18:11:40.452 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49410.log 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72638.log 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49410.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49410.log.gz 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72638.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72638.log.gz 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48595.log 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56763.log 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48595.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48595.log.gz 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56763.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56763.log.gz 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-04-24T18:11:40.453 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73629.log 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mon.c.log: /var/log/ceph/ceph-client.admin.73629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73629.log.gz 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54353.log 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62873.log 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54353.log.gz 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62873.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62873.log.gz 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58162.log 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58162.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58162.log.gz 2024-04-24T18:11:40.454 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55612.log 2024-04-24T18:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55612.log.gz 2024-04-24T18:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57807.log 2024-04-24T18:11:40.455 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57807.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57807.log.gz 2024-04-24T18:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67570.log 2024-04-24T18:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67570.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67570.log.gz 2024-04-24T18:11:40.456 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50695.log 2024-04-24T18:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50695.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50695.log.gz 2024-04-24T18:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59742.log 2024-04-24T18:11:40.457 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52868.log 2024-04-24T18:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59742.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59742.log.gz 2024-04-24T18:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52868.log.gz 2024-04-24T18:11:40.458 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48249.log 2024-04-24T18:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48249.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48249.log.gz 2024-04-24T18:11:40.459 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53727.log 2024-04-24T18:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53727.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53727.log.gz 2024-04-24T18:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73951.log 2024-04-24T18:11:40.460 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56939.log 2024-04-24T18:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73951.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73951.log.gz 2024-04-24T18:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56939.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56939.log.gz 2024-04-24T18:11:40.461 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63746.log 2024-04-24T18:11:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63746.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63746.log.gz 2024-04-24T18:11:40.462 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72266.log 2024-04-24T18:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72266.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72266.log.gz 2024-04-24T18:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65935.log 2024-04-24T18:11:40.463 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67416.log 2024-04-24T18:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65935.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65935.log.gz 2024-04-24T18:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67416.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67416.log.gz 2024-04-24T18:11:40.464 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73675.log 2024-04-24T18:11:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73675.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73675.log.gz 2024-04-24T18:11:40.465 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57414.log 2024-04-24T18:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57414.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57414.log.gz 2024-04-24T18:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64902.log 2024-04-24T18:11:40.466 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.64902.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64902.log.gz 2024-04-24T18:11:40.467 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48351.log 2024-04-24T18:11:40.468 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48351.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48351.log.gz 2024-04-24T18:11:40.468 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73215.log 2024-04-24T18:11:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73215.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73215.log.gz 2024-04-24T18:11:40.470 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54992.log 2024-04-24T18:11:40.471 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54992.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54992.log.gz 2024-04-24T18:11:40.471 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74227.log 2024-04-24T18:11:40.472 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74227.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74227.log.gz 2024-04-24T18:11:40.472 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71375.log 2024-04-24T18:11:40.472 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71375.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71375.log.gz 2024-04-24T18:11:40.473 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66868.log 2024-04-24T18:11:40.473 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66868.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66868.log.gz 2024-04-24T18:11:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56064.log 2024-04-24T18:11:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56064.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56064.log.gz 2024-04-24T18:11:40.474 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56395.log 2024-04-24T18:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56395.log.gz 2024-04-24T18:11:40.475 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54168.log 2024-04-24T18:11:40.476 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54168.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54168.log.gz 2024-04-24T18:11:40.476 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48302.log 2024-04-24T18:11:40.479 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48302.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48302.log.gz 2024-04-24T18:11:40.479 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47604.log 2024-04-24T18:11:40.480 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47604.log.gz 2024-04-24T18:11:40.480 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66521.log 2024-04-24T18:11:40.481 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66521.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66521.log.gz 2024-04-24T18:11:40.481 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59855.log 2024-04-24T18:11:40.481 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59855.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59855.log.gz 2024-04-24T18:11:40.482 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71651.log 2024-04-24T18:11:40.482 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71651.log.gz 2024-04-24T18:11:40.482 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57784.log 2024-04-24T18:11:40.483 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57784.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57784.log.gz 2024-04-24T18:11:40.483 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71132.log 2024-04-24T18:11:40.484 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71132.log.gz 2024-04-24T18:11:40.484 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60649.log 2024-04-24T18:11:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60649.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60649.log.gz 2024-04-24T18:11:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49776.log 2024-04-24T18:11:40.485 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60246.log 2024-04-24T18:11:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49776.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49776.log.gz 2024-04-24T18:11:40.486 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74273.log 2024-04-24T18:11:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60246.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60246.log.gz 2024-04-24T18:11:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74273.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74273.log.gz 2024-04-24T18:11:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53823.log 2024-04-24T18:11:40.487 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53823.log.gz 2024-04-24T18:11:40.488 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52623.log 2024-04-24T18:11:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52623.log.gz 2024-04-24T18:11:40.489 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68171.log 2024-04-24T18:11:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68171.log.gz 2024-04-24T18:11:40.490 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67756.log 2024-04-24T18:11:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67756.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67756.log.gz 2024-04-24T18:11:40.491 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72626.log 2024-04-24T18:11:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72626.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72626.log.gz 2024-04-24T18:11:40.492 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61112.log 2024-04-24T18:11:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61112.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61112.log.gz 2024-04-24T18:11:40.493 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50602.log 2024-04-24T18:11:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50602.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50602.log.gz 2024-04-24T18:11:40.494 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70188.log 2024-04-24T18:11:40.495 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70188.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70188.log.gz 2024-04-24T18:11:40.495 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66914.log 2024-04-24T18:11:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.66914.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66914.log.gz 2024-04-24T18:11:40.496 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54191.log 2024-04-24T18:11:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54191.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54191.log.gz 2024-04-24T18:11:40.497 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64101.log 2024-04-24T18:11:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.64101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64101.log.gz 2024-04-24T18:11:40.498 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60105.log 2024-04-24T18:11:40.499 INFO:teuthology.orchestra.run.smithi078.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-24T18:11:40.499 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60105.log.gz 2024-04-24T18:11:40.499 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73813.log 2024-04-24T18:11:40.500 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73813.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73813.log.gz 2024-04-24T18:11:40.500 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55037.log 2024-04-24T18:11:40.501 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55037.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55037.log.gz 2024-04-24T18:11:40.501 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56890.log 2024-04-24T18:11:40.502 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56890.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56890.log.gz 2024-04-24T18:11:40.502 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49364.log 2024-04-24T18:11:40.503 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49364.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49364.log.gz 2024-04-24T18:11:40.503 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69411.log 2024-04-24T18:11:40.504 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69411.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69411.log.gz 2024-04-24T18:11:40.504 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70785.log 2024-04-24T18:11:40.505 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70785.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70785.log.gz 2024-04-24T18:11:40.505 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73997.log 2024-04-24T18:11:40.506 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73997.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73997.log.gz 2024-04-24T18:11:40.506 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52317.log 2024-04-24T18:11:40.507 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.52317.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52317.log.gz 2024-04-24T18:11:40.507 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71881.log 2024-04-24T18:11:40.508 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71881.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71881.log.gz 2024-04-24T18:11:40.508 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67260.log 2024-04-24T18:11:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.67260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67260.log.gz 2024-04-24T18:11:40.509 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54791.log 2024-04-24T18:11:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54791.log.gz 2024-04-24T18:11:40.510 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48593.log 2024-04-24T18:11:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48593.log.gz 2024-04-24T18:11:40.511 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74365.log 2024-04-24T18:11:40.512 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74365.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74365.log.gz 2024-04-24T18:11:40.512 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71927.log 2024-04-24T18:11:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71927.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71927.log.gz 2024-04-24T18:11:40.513 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63101.log 2024-04-24T18:11:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63101.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63101.log.gz 2024-04-24T18:11:40.514 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68041.log 2024-04-24T18:11:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68041.log.gz 2024-04-24T18:11:40.515 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49774.log 2024-04-24T18:11:40.516 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49774.log.gz 2024-04-24T18:11:40.516 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61183.log 2024-04-24T18:11:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61183.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61183.log.gz 2024-04-24T18:11:40.517 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58057.log 2024-04-24T18:11:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.58057.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58057.log.gz 2024-04-24T18:11:40.518 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48594.log 2024-04-24T18:11:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48594.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48594.log.gz 2024-04-24T18:11:40.519 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51197.log 2024-04-24T18:11:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51197.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51197.log.gz 2024-04-24T18:11:40.520 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71421.log 2024-04-24T18:11:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71421.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71421.log.gz 2024-04-24T18:11:40.521 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56740.log 2024-04-24T18:11:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56740.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56740.log.gz 2024-04-24T18:11:40.522 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-04-24T18:11:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-mgr.y.log: 93.3% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-04-24T18:11:40.529 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62719.log 2024-04-24T18:11:40.530 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62719.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62719.log.gz 2024-04-24T18:11:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43604.log 2024-04-24T18:11:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.43604.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43604.log.gz 2024-04-24T18:11:40.531 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53598.log 2024-04-24T18:11:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53598.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53598.log.gz 2024-04-24T18:11:40.532 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47949.log 2024-04-24T18:11:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47949.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47949.log.gz 2024-04-24T18:11:40.533 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49226.log 2024-04-24T18:11:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49226.log.gz 2024-04-24T18:11:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71283.log 2024-04-24T18:11:40.534 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71283.log.gz 2024-04-24T18:11:40.535 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73583.log 2024-04-24T18:11:40.536 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73583.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73583.log.gz 2024-04-24T18:11:40.536 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62196.log 2024-04-24T18:11:40.536 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62196.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62196.log.gz 2024-04-24T18:11:40.536 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49456.log 2024-04-24T18:11:40.537 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49456.log.gz 2024-04-24T18:11:40.537 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.74181.log 2024-04-24T18:11:40.538 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.74181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.74181.log.gz 2024-04-24T18:11:40.538 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56439.log 2024-04-24T18:11:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.56439.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56439.log.gz 2024-04-24T18:11:40.539 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73353.log 2024-04-24T18:11:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73353.log.gz 2024-04-24T18:11:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63406.log 2024-04-24T18:11:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63406.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63406.log.gz 2024-04-24T18:11:40.540 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49134.log 2024-04-24T18:11:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.49134.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49134.log.gz 2024-04-24T18:11:40.541 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59490.log 2024-04-24T18:11:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59490.log.gz 2024-04-24T18:11:40.542 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72629.log 2024-04-24T18:11:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72629.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72629.log.gz 2024-04-24T18:11:40.543 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53462.log 2024-04-24T18:11:40.544 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53462.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53462.log.gz 2024-04-24T18:11:40.544 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69641.log 2024-04-24T18:11:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69641.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69641.log.gz 2024-04-24T18:11:40.545 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71012.log 2024-04-24T18:11:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71012.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71012.log.gz 2024-04-24T18:11:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55456.log 2024-04-24T18:11:40.546 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55456.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55456.log.gz 2024-04-24T18:11:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61983.log 2024-04-24T18:11:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.61983.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61983.log.gz 2024-04-24T18:11:40.547 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69271.log 2024-04-24T18:11:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69271.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69271.log.gz 2024-04-24T18:11:40.548 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53072.log 2024-04-24T18:11:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.53072.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53072.log.gz 2024-04-24T18:11:40.549 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60106.log 2024-04-24T18:11:40.550 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.60106.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60106.log.gz 2024-04-24T18:11:40.550 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72267.log 2024-04-24T18:11:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72267.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72267.log.gz 2024-04-24T18:11:40.551 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68651.log 2024-04-24T18:11:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.68651.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68651.log.gz 2024-04-24T18:11:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54608.log 2024-04-24T18:11:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54608.log.gz 2024-04-24T18:11:40.552 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62827.log 2024-04-24T18:11:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.62827.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62827.log.gz 2024-04-24T18:11:40.553 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51563.log 2024-04-24T18:11:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51563.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51563.log.gz 2024-04-24T18:11:40.554 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73767.log 2024-04-24T18:11:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73767.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73767.log.gz 2024-04-24T18:11:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59918.log 2024-04-24T18:11:40.555 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54628.log 2024-04-24T18:11:40.556 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.59918.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59918.log.gz 2024-04-24T18:11:40.556 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54628.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54628.log.gz 2024-04-24T18:11:40.556 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48526.log 2024-04-24T18:11:40.558 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.48526.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48526.log.gz 2024-04-24T18:11:40.558 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.73491.log 2024-04-24T18:11:40.559 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.73491.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.73491.log.gz 2024-04-24T18:11:40.559 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70143.log 2024-04-24T18:11:40.560 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.70143.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70143.log.gz 2024-04-24T18:11:40.560 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47942.log 2024-04-24T18:11:40.561 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.47942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47942.log.gz 2024-04-24T18:11:40.561 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.71467.log 2024-04-24T18:11:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.71467.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.71467.log.gz 2024-04-24T18:11:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54789.log 2024-04-24T18:11:40.562 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54789.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54789.log.gz 2024-04-24T18:11:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72608.log 2024-04-24T18:11:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72608.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72608.log.gz 2024-04-24T18:11:40.563 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.72065.log 2024-04-24T18:11:40.564 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.72065.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.72065.log.gz 2024-04-24T18:11:40.564 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50231.log 2024-04-24T18:11:40.565 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.50231.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50231.log.gz 2024-04-24T18:11:40.565 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65286.log 2024-04-24T18:11:40.566 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65286.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65286.log.gz 2024-04-24T18:11:40.566 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63514.log 2024-04-24T18:11:40.567 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.63514.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63514.log.gz 2024-04-24T18:11:40.567 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69987.log 2024-04-24T18:11:40.567 INFO:teuthology.orchestra.run.smithi078.stderr: 90.9% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-04-24T18:11:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.69987.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69987.log.gz 2024-04-24T18:11:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54946.log 2024-04-24T18:11:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.54946.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54946.log.gz 2024-04-24T18:11:40.568 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57531.log 2024-04-24T18:11:40.569 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.57531.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57531.log.gz 2024-04-24T18:11:40.569 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65454.log 2024-04-24T18:11:40.570 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.65454.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65454.log.gz 2024-04-24T18:11:40.570 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51255.log 2024-04-24T18:11:40.571 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.51255.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51255.log.gz 2024-04-24T18:11:40.571 INFO:teuthology.orchestra.run.smithi049.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55105.log 2024-04-24T18:11:40.572 INFO:teuthology.orchestra.run.smithi049.stderr:/var/log/ceph/ceph-client.admin.55105.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55105.log.gz 2024-04-24T18:11:40.627 INFO:teuthology.orchestra.run.smithi049.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-04-24T18:11:40.717 INFO:teuthology.orchestra.run.smithi049.stderr: 90.4% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-24T18:11:41.051 INFO:teuthology.orchestra.run.smithi078.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-04-24T18:11:41.811 INFO:teuthology.orchestra.run.smithi049.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-24T18:11:42.136 INFO:teuthology.orchestra.run.smithi078.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-04-24T18:11:42.967 INFO:teuthology.orchestra.run.smithi078.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-04-24T18:11:44.097 INFO:teuthology.orchestra.run.smithi049.stderr: 95.3% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-24T18:11:44.173 INFO:teuthology.orchestra.run.smithi049.stderr: 95.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-24T18:11:45.892 INFO:teuthology.orchestra.run.smithi049.stderr: 95.4% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-24T18:11:45.894 INFO:teuthology.orchestra.run.smithi049.stderr: 2024-04-24T18:11:45.894 INFO:teuthology.orchestra.run.smithi049.stderr:real 0m5.539s 2024-04-24T18:11:45.894 INFO:teuthology.orchestra.run.smithi049.stderr:user 0m14.440s 2024-04-24T18:11:45.894 INFO:teuthology.orchestra.run.smithi049.stderr:sys 0m0.700s 2024-04-24T18:11:47.577 INFO:teuthology.orchestra.run.smithi078.stderr: 95.6% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-04-24T18:11:47.579 INFO:teuthology.orchestra.run.smithi078.stderr: 2024-04-24T18:11:47.579 INFO:teuthology.orchestra.run.smithi078.stderr:real 0m7.189s 2024-04-24T18:11:47.579 INFO:teuthology.orchestra.run.smithi078.stderr:user 0m11.879s 2024-04-24T18:11:47.579 INFO:teuthology.orchestra.run.smithi078.stderr:sys 0m0.531s 2024-04-24T18:11:47.579 INFO:tasks.ceph:Archiving logs... 2024-04-24T18:11:47.579 DEBUG:teuthology.misc:Transferring archived files from smithi049:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/remote/smithi049/log 2024-04-24T18:11:47.580 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T18:11:49.502 DEBUG:teuthology.misc:Transferring archived files from smithi078:/var/log/ceph to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/remote/smithi078/log 2024-04-24T18:11:49.503 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-24T18:11:51.005 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-24T18:11:51.015 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-24T18:11:51.015 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T18:11:51.017 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-24T18:11:51.100 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-24T18:11:51.109 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-24T18:11:51.109 DEBUG:teuthology.orchestra.run.smithi049:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T18:11:51.112 DEBUG:teuthology.orchestra.run.smithi078:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-24T18:11:51.128 INFO:teuthology.orchestra.run.smithi049.stderr:bash: ntpq: command not found 2024-04-24T18:11:51.129 INFO:teuthology.orchestra.run.smithi078.stderr:bash: ntpq: command not found 2024-04-24T18:11:51.133 INFO:teuthology.orchestra.run.smithi049.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T18:11:51.133 INFO:teuthology.orchestra.run.smithi049.stdout:=============================================================================== 2024-04-24T18:11:51.133 INFO:teuthology.orchestra.run.smithi049.stdout:^- hv01.front.sepia.ceph.com 2 6 377 47 +1429us[+1423us] +/- 582ms 2024-04-24T18:11:51.133 INFO:teuthology.orchestra.run.smithi049.stdout:^* hv02.front.sepia.ceph.com 3 6 377 46 +194us[ +188us] +/- 55ms 2024-04-24T18:11:51.133 INFO:teuthology.orchestra.run.smithi049.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 45 -179us[ -179us] +/- 65ms 2024-04-24T18:11:51.133 INFO:teuthology.orchestra.run.smithi049.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T18:11:51.135 INFO:teuthology.orchestra.run.smithi078.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-24T18:11:51.135 INFO:teuthology.orchestra.run.smithi078.stdout:=============================================================================== 2024-04-24T18:11:51.135 INFO:teuthology.orchestra.run.smithi078.stdout:^- hv01.front.sepia.ceph.com 2 6 377 46 +1423us[+1474us] +/- 582ms 2024-04-24T18:11:51.135 INFO:teuthology.orchestra.run.smithi078.stdout:^* hv02.front.sepia.ceph.com 3 6 377 45 +184us[ +235us] +/- 55ms 2024-04-24T18:11:51.135 INFO:teuthology.orchestra.run.smithi078.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 -182us[ -130us] +/- 65ms 2024-04-24T18:11:51.135 INFO:teuthology.orchestra.run.smithi078.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-24T18:11:51.135 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-24T18:11:51.144 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-24T18:11:51.144 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-24T18:11:51.200 DEBUG:teuthology.orchestra.run.smithi049:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T18:11:51.291 DEBUG:teuthology.orchestra.run.smithi078:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-24T18:11:51.371 DEBUG:teuthology.orchestra.run.smithi049:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T18:11:51.404 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T18:11:51.404 DEBUG:teuthology.orchestra.run.smithi078:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2024-04-24T18:11:51.435 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T18:11:51.436 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-24T18:11:51.445 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-24T18:11:51.454 INFO:teuthology.task.internal:Duration was 1539.439028 seconds 2024-04-24T18:11:51.454 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-24T18:11:51.463 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-24T18:11:51.463 DEBUG:teuthology.orchestra.run.smithi049:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T18:11:51.466 DEBUG:teuthology.orchestra.run.smithi078:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-24T18:11:51.513 INFO:teuthology.orchestra.run.smithi049.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T18:11:51.524 INFO:teuthology.orchestra.run.smithi078.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-24T18:11:51.751 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-24T18:11:51.794 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi049.front.sepia.ceph.com 2024-04-24T18:11:51.794 DEBUG:teuthology.orchestra.run.smithi049:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T18:11:51.816 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi078.front.sepia.ceph.com 2024-04-24T18:11:51.817 DEBUG:teuthology.orchestra.run.smithi078:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-24T18:11:51.840 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-24T18:11:51.840 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T18:11:51.860 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-24T18:11:51.927 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-24T18:11:51.928 DEBUG:teuthology.orchestra.run.smithi049:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T18:11:51.930 DEBUG:teuthology.orchestra.run.smithi078:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-24T18:11:52.035 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-24T18:11:52.044 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-24T18:11:52.044 DEBUG:teuthology.orchestra.run.smithi049:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T18:11:52.072 DEBUG:teuthology.orchestra.run.smithi078:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-24T18:11:52.099 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-24T18:11:52.110 DEBUG:teuthology.orchestra.run.smithi049:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T18:11:52.114 DEBUG:teuthology.orchestra.run.smithi078:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-24T18:11:52.139 INFO:teuthology.orchestra.run.smithi049.stdout:kernel.core_pattern = core 2024-04-24T18:11:52.165 INFO:teuthology.orchestra.run.smithi078.stdout:kernel.core_pattern = core 2024-04-24T18:11:52.217 DEBUG:teuthology.orchestra.run.smithi049:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T18:11:52.233 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T18:11:52.233 DEBUG:teuthology.orchestra.run.smithi078:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-24T18:11:52.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-24T18:11:52.274 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-24T18:11:52.284 INFO:teuthology.task.internal:Transferring archived files... 2024-04-24T18:11:52.285 DEBUG:teuthology.misc:Transferring archived files from smithi049:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/remote/smithi049 2024-04-24T18:11:52.285 DEBUG:teuthology.orchestra.run.smithi049:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T18:11:52.332 DEBUG:teuthology.misc:Transferring archived files from smithi078:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2024-04-24_14:09:24-rbd-wip-yuriw4-testing-20240423.151325-quincy-distro-default-smithi/7671641/remote/smithi078 2024-04-24T18:11:52.332 DEBUG:teuthology.orchestra.run.smithi078:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-24T18:11:52.370 INFO:teuthology.task.internal:Removing archive directory... 2024-04-24T18:11:52.371 DEBUG:teuthology.orchestra.run.smithi049:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T18:11:52.373 DEBUG:teuthology.orchestra.run.smithi078:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-24T18:11:52.420 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-24T18:11:52.431 INFO:teuthology.task.internal:Not uploading archives. 2024-04-24T18:11:52.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-24T18:11:52.441 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-24T18:11:52.441 DEBUG:teuthology.orchestra.run.smithi049:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T18:11:52.443 DEBUG:teuthology.orchestra.run.smithi078:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-24T18:11:52.458 INFO:teuthology.orchestra.run.smithi049.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 18:11 /home/ubuntu/cephtest 2024-04-24T18:11:52.476 INFO:teuthology.orchestra.run.smithi078.stdout: 262175 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 24 18:11 /home/ubuntu/cephtest 2024-04-24T18:11:52.477 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-24T18:11:52.488 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-24T18:11:52.510 INFO:teuthology.run:Summary data: description: rbd/thrash/{base/install clusters/{fixed-2 openstack} conf/{disable-pool-app} msgr-failures/few objectstore/bluestore-comp-lz4 supported-random-distro$/{rhel_8} thrashers/cache thrashosds-health workloads/rbd_fsx_cache_writearound} duration: 1539.4390277862549 flavor: default owner: scheduled_yuriw@teuthology success: true 2024-04-24T18:11:52.510 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-24T18:11:52.583 INFO:teuthology.run:pass